The following information is provided for users, developers, and new reviewers of MERLOT-Engineering materials to help them better understand the criteria used to evaluate materials in the database. The criteria used in the peer review process are intended to help users select materials which meet their needs and to aid authors in the development of quality materials for the MERLOT database.

There are three general categories of evaluation standards:

1. Quality of Content
2. Potential Effectiveness as a Teaching-Learning Tool
3. Ease of Use

Though the three categories are interdependent to some extent, separate ratings are assigned to each on a 1 to 5 scale. Materials are given a rating of 3 if they are considered useful at a basic level. Materials that satisfy two or three of the main criteria in each evaluation category are given a rating of 4, and materials that satisfy most of the main criteria are given a rating of 5.

Reviews with ratings of 3 or above are posted on MERLOT. Reviews with lower ratings are not posted, but are sent back to the authors for their information and potential revision.

At least two reviewers are assigned to each material reviewed. Each reviewer performs an individual review, discusses it with the other reviewer, and together they produce a final composite version. Authors see all reviews before they are posted and have an opportunity to respond to the review or to request that it not be posted.

QUALITY OF CONTENT

1. Does the material present valid concepts, models, collections, and/or results?

2. Does the material present important engineering concepts, models, or collections?

a. Is the content part of the core curriculum in the specific engineering discipline?
b. Is the content a prerequisite for more advanced material in the discipline?
c. Does the material stay on target?
d. Does the material address topics that appear to be difficult to teach?
e. Does the material address topics that appear to be difficult to learn?

3. Does the material make effective use of graphics and multimedia?

a. Are the graphics or media attractive?
b. Do the graphics or media contribute to a better understanding of the engineering.

 

POTENTIAL EFFECTIVENESS AS A TEACHING-LEARNING TOOL

1. Does the material improve faculty's abilities to teach and students' abilities to learn the concepts it addresses?

a) Does the material effectively demonstrate the engineering concepts it addresses?
b) Does the material make difficult engineering concepts more understandable?
c) Is the material a learning tool and not just an entertaining diversion?
d) Does the material help develop problem solving skills?

2. Can the material be used in a variety of ways to achieve teaching and learning goals?

a) Can the material be readily used in the classroom to supplement lectures?
b) Can the material be readily incorporated into homework assignments?
c) Can the material be readily used for group assignments?
d) Can the material be readily used for laboratory assignments?
e) Can the material be readily used in a tutorial fashion?

3. Can the material be readily integrated into an engineering curriculum?

a) Does the material enhance standard presentations of engineering concepts?
b) Can the material be used with standard textbooks or problems?
c) Can the material be used in a nontraditional manner?

4. Are the teaching-learning goals and consequent uses easy to identify?

a) Are the teaching-learning goals readily identifiable?
b) Are uses for the material readily identifiable?

5. Can good learning assignments that use the material be written easily?

EASE OF USE

1. Is the material convenient and inviting to use?

a) Is the software easy to enter and exit?
b) Is the material attractive and inviting to use?
c) Would someone likely become lost or confused while using the material?
d) Can the material be used with minimal documentation or instruction?
e) Is the software robust enough to minimize the risk of a crash?
f) Is the background of the expected user clearly indicated?

2. Is the layout of the material consistent and intuitive?

a) Does the material present information in ways that are familiar to engineering faculty and students?
b) Are labels, buttons, and other interface elements fully functional, consistent, and intuitive?

3. Does the software provide appropriate feedback?

a) Is appropriate feedback provided about system status and user responses?
b) Is feedback sufficiently rapid to be useful?

4. Does the software provide appropriate flexibility in its use?

a) Can the material be used by faculty and students with different levels of experience with computers and the web?
b) Is the material adaptable to different internet connection speeds?
c) Is the material free of any special software requirements such as difficult to obtain plug-ins, media players, browser type, etc.?

--- Revised February 4, 2002 ---