[Next] [Previous] [Top] [Contents]

A Proposal for the Establishment of Review Boards - Bruce Edmonds

3. A Proposal for the Establishment of Electronic Review Boards


The basic idea is that review board will be set up which will review on-line papers (whether on home pages or in archives). The reviews will be done by a board of reviewers, who would aim to cover a particular area of knowledge (and maybe only a particular aspect of such an area). The reviews would consist of judgements of the papers in a number of different ways. A paper could be given a grade from one to five on presentation, relevance, soundness of argument, originality, importance of questions considered, importance of results etc. as well as short comments about the paper for public consumption. These reviews would be submitted electronically and automatically collated into a set of publicly accessible records. These records would include information about the content (title, author, abstract, keywords etc.) as well as the compiled qualitative judgements (number or reviewers, average grades, comments etc.).

These records would be accessible to readers using a variety of interfaces, but including a search engine. This means that readers could search for information using a mixture of judgemental and content-based information. For instance: they could look for all papers on a subject which were judged to have very important results, so as to keep in touch with important developments; or they could look for papers with high originality however bad which mention a keyword if they needed some new ideas. In this way the reader can avoid being swamped by information with irrelevant characteristics and be assured of the quality of what is accessed without needlessly restricting the flow of information at an early a stage. The flow of quality information is not restricted in a generic way earlier in the publishing process, but instead the information allowing the later constraint of information is made available to readers so that they can constrain the information in the way that suits their needs.

At the moment readers are faced with a choice, either to read journals whose content will not be selected according to their precise needs or to select from the web or public archive on content, but have to wade through papers of low quality in ways that are important to the reader. The traditional system is illustrated in figure 1.

Figure 1. The existing journal and archive systems

Alternatively, the proposed system delays the selection process as long a possible and thus does not discard any information which may turn out to be of use later, but provides the user with the means to exclude the mass of papers in a way which meets the reader's needs at that time. The proposed system is illustrated in figure 2.

Figure 2. The proposed system

Of course the above illustrations are give a somewhat over-simple picture of the processes. There is a variety of journals, of different standards selecting on different purposes. Readers can use implicit and explicit knowledge of the quality and coverage of a journal to aid their search process. I presume that, if review boards were set up then there would be at least as great a variety of boards as there are presently of journals. Comparing like-with-like the journal system imposes premature, inflexible and unnecessary restrictions upon the flow of knowledge compared with a system of review boards.

The fact is that at the moment the only practical and general way to avoid scanning papers whose qualities are insufficient for one's purposes is to accept the selection of a journal whose criteria will not be identical to one's own. Of course, if one is in the happy position of having access to a manageable sized set of journals which (collectively) cover all ones needs then the journal system is sufficient, but many readers are not in such a position.

There is no reason why such a board should necessarily choose to restrict itself to those papers that are submitted to it by authors. It could equally well choose to review any papers that would be relevant to its readers (although if it were a paper published in a paper journal it would probably not be possible to supply an on-line link to it). Thus a review board could not only review new papers for its readers but also provide a new view of existing papers.

A system of review that is close to the above suggestions has bee made


A Proposal for the Establishment of Review Boards - Bruce Edmonds - 16 MAR 99
[Next] [Previous] [Top] [Contents]

Generated with CERN WebMaker