LMS Evaluation Checklist

Section 1. FUNCTIONALITY OF LMS SOFTWARE FOR DEVELOPERS & INSTRUCTORS
This section covers the academic and instructional design functionality of the proposed learning management system.
Assessments/Assignments/Grade Book Priority  Score  Comments
Ability of LMS to allow browser lockdown techniques to ensure assessment security.
Ability of LMS to allow proctor passwords to ensure assessment security.
Ability of LMS to allow IP restrictions to ensure assessment can only be taken in certain locations (campus lab).
Ability of testing tool to randomize quiz question answers.
Ability of testing tool to randomize quiz questions.
Ability to embed videos into assessments.
Ability to except student Web pages and collaborative projects with multimedia.
Ability to integrate course content with rubrics.
Ability to send grades from LMS to Peoplesoft.
Ability to set conditions (i.e., hide/show, dates) for multiple assessments at the same time.
Ability to use rubrics to grade essay questions within an online assessment.
Ability to copy rubrics from semester to semester
Ability to use one rubric in multiple courses sections
Assignment Submissions: PDF
Assignment Submissions: Video
Assignment Submissions: Web Page
Assignment Submissions: Word Doc
Assignment Submissions: Zip File
Peer review tools are available.
Quiz Question Type: Fill in the Blank/Short Answer
Quiz Question Type: Long Answer/Paragraph
Quiz Question Type: Matching
Quiz Question Type: Multiple Choice
Quiz Question Type: True or False
Quiz Question Type: Word Scramble
Ability to create graded pre and post-test surveys
Group management features available for creating group assignments
Ability to create a group discussion topic or assignment and then assign it to all groups at once
Ability to provide detailed tracking of when students select an answer (i.e. time stamp of answer choice) within the Assessment tool
Ability to re-grade assessments
Ability to grade objective quizzes automatically and reveals results to students.
Ability to selectively release assessments based on date.
Ability to selectively release assessments based on time.
Ability to selectively release assessments based on student ID.
Ability to selectively release assessments based on grade book criteria.
Ability to selectively release assessments based on multiple criteria.
Ability for students to submit papers online and allow instructors to grade and give feedback without downloading the assignment.
Ability to see student submission dates for an assignment in the grade book.
Ability for students to see their grades (MyGrades Area).
Ability to track to easily track attendance in the Grade Book (example: last login date).
Ability to notify/alerts at-risk students.
Ability to export and import from Excel into the Grade Book.
Ability to include or support the use of portfolios to document student learning.
Ability of system to support statistical analysis of assessment results.
Self-assessments are available.
Ability to change the file size limit on files that can be submitted to the Assignment Tool.
Ability of testing tool to allow the use of math symbols.
Ability of testing tool to allow the use of images.
Ability of assessments be exported to another course.
Ability to map learning outcomes to assessments in a way that achievement of specific outcomes can be evaluated.
Announcements Priority  Score Comments
Ability for students to receive announcements outside the LMS for emergency notifications.
Ability for students to view announcements within the LMS.
Ability to copy announcements to new courses.
Ability for System Admins can edit active (system) announcements.
Ability for System Admins to edit/delete any role’s institution announcements.
Ability for Instructors can edit active announcements.
Blogs Priority  Score Comments
LMS has a Blog tool.
Ability to use the WYSIWIG or HTML editor within the Blog tool.
Bookmarks Priority  Score Comments
Ability for instructors to bookmark areas within the course.
Ability for students to bookmark areas within the course.
Calendar Priority  Score Comments
Ability to set start and stop time for calendar events.
Ability to create a reoccurring calendar entries
Ability to provide an RSS feed for students to subscribe
Ability for users to subscribe to an external calendar
Ability for calendar events to be added automatically from any tool that has a due date
Copying Content Priority  Score Comments
Ability for instructors to easily copy (clone) from an existing section to a blank section.
Ability for instructors to select specific items they would like to copy into a new course
Ability to share content between courses.
Ability to easily package a course to import into other LMS platforms or repositories.
Chat and Whiteboard Priority  Score Comments
LMS has a live chat feature.
LMS has a live whiteboard feature.
Ability to log/save chat transcripts.
Ability of whiteboard to access/use an equation editor.
Community Networking Priority  Score Comments
Ability to use RSS to send notifications of updated content to students.
Ability to include social media integration.
Course Templates Priority  Score Comments
Ability to create course templates
Ability to assign ownership of course templates to a selected group of users
Ability to assign course templates via the SIS integration
Discussion Forums Priority  Score Comments
Threaded discussions are allowed.
Ability to grade discussion postings
Ability for students to attach documents in discussion forums.
Ability for students to embed links in discussion forums.
Ability to allow group creation in discussion forums.
Ability to have anonymous postings.
Ability to set release conditions (i.e., hide/show, dates) for multiple discussions at the same time.
Ability to search Discussions
Ability to moderate discussion forums
Ability for students to have private discussions that cannot be viewed by the teacher.
Ability to use rubrics to grade discussion forums.
Ability to use the WYSIWIG or HTML editor in the discussion forums.
IM Enabled Priority  Score Comments
Ability for students and faculty see who is logged into courses.
Ability for IM to permit file sharing.
Ability for IM to permit two-way video.
Built-in instant messaging capability.
Ability for students and faculty to use IM to communicate.
Ability for user to hide his/her presence in the IM tool.
Internal E-mail Priority  Score Comments
E-mail tool provides “spell checking” ability for narrative fields.
Ability to set up rules within the e-mail tool.
Ability to administratively set students’ LMS email address to a campus standard system for all students in all courses.
Ability to forward internal e-mail to an external account.
Ability to use the WYSIWIG or HTML editor in the internal e-mail tool.
Ability to archive incoming and outgoing e-mails communications throughout the course.
Ability to use search capabilities in the internal e-mail.
Ability to create folders for filing/sorting e-mails within the e-mail tool.
Mashups Priority  Score Comments
Ability to enable learner and instructor mashups.
Mobility Priority  Score Comments
Ability to use mobile devices to access course content.
All course tools are available in the mobile client
Access to the LMS’s mobile client is standard.
There is a pricing plan for mobile client access to the LMS.
Available tools in the mobile client have the same functionality as the web version.
The LMS is mobile-ready when using the web version through smart phones.
The LMS is mobile-ready when using the web version through tablets.
Online Journal Priority  Score Comments
LMS have a journal tool.
Ability to use the WYSIWIG or HTML editor in the journal tool.
Online Marking Tools/Notes Priority  Score Comments
Ability for students to make notations within the course.
Search Functionality Priority  Score Comments
Ability to use search within a course to find course content.
SMS Messaging Priority  Score Comments
Ability of LMS to send text message alerts.
Social Media Integration Priority  Score Comments
Ability to enable students to create social connections.
Ability to integrate with Facebook.
Ability to integrate with Twitter.
Ability to integrate with YouTube.
Student Portfolios Priority  Score Comments
Ability to access leading portfolio solutions through single sign-on
LMS has a built-in portfolio solution
Ability for students to access e-portfolios after graduation
Ability to provide access to e-portfolios to students after graduation for free
The portfolio remains active for a reasonable period of time
What options does the user have to remain active after the period of time has expired?
Ability to view discrete parts of the portfolio publicly
What is the storage limitation per student and how flexible is the system for expansion?
Student Tracking Priority  Score Comments
Is detailed student system tracking available (log in, log out, last accessed, IP)
Is detailed student tracking available for each course tool (How many discussion posts were read, how long did they spend on each quiz question)
Can the tracking schedule be changed by System Admins
Can tracking jobs be monitored from a GUI interface
Third Party Test Management Priority  Score Comments
Does your LMS support browser lockdown applications, like Respondus Lockdown Browser?
Ability to import and export items in QTI (IMS Question and Test Interoperability).
Wikis Priority  Score Comments
LMS has a Wiki tool.
How does an instructor create wikis within the course?
Describe the functionality of the wiki tool (e.g. versioning, HTML editor).
WYSIWYG Editing Priority  Score Comments
LMS has a built in fully functional WYSISYG or HTML editor that is available for students and faculty.
LMS provides a fully functional equation editor.
WYSISYG or HTML editor can modify HTML pages created outside of the LMS.
The WYSIWIG allows you to apply formatting styles (e.g., heading styles, bulleted/numbered lists)
Does your LMS provide a drag and drop, customizable environment (for example, utilizes technologies such as AJAX)?
Can your LMS accommodate custom banners and icons on course homepage, header, footers, etc.
Identify and detail any features not mentioned in this section that you wish to be considered.
Section 2. FUNCTIONALITY OF LMS SOFTWARE FOR ADMINSTRATION
This section covers the administrative interface and capabilities of the proposed system, as well as the level of compliance with interoperability and usability standards.
Accessibility Priority Score Comments
ADA/Section 508 compliance requirements are followed.
Ability to support the use of screen readers (example: JAWS).
Ability to set different assessment time frames for individual students (e.g., students with disabilities who need more time)
Ability to provide a variety of feedback types (e.g., text, audio, video) for assignments, discussions, other student activities
Ability to add audio/video components to quizzes or exams.
Ability to add captions to assessments
Authentication at Industry Standards Priority  Score Comments
Ability to support SSO from a PeopleSoft Portal.
Ability to support authentication using LDAP.
Ability to support authentication using Shibboleth.
Ability to support authentication using Kerberos.
Ability to support external and internal authentication.
SSO SAML (Security Assertion Markup Language) is compliant.
Branding Flexibility Priority  Score Comments
Ability to brand as Webcourses@UCF instance.
Combine Sections Priority  Score Comments
Ability for faculty to combine section
Ability for faculty to combine sections not listed under the same college
Ability for System Admins to un-combine sections via the GUI
Ability to combine sections via the SIS integration
Concurrent Users Priority  Score Comments
Concurrent user capabilities
Ability to scale the implementation for increased usage.
Course cloning and archive restoration Priority  Score Comments
Ability to handle backup and retrieval options at the system administrator level.
Ability to use command line to create backups in batch.
Ability to use GUI to create backups in batch.
Ability for Blackboard Vista .bak archives to be restored in the LMS with student.
Ability for Blackboard Vista .bak archives to be restored in the LMS without student.
Ability to restore student artifacts in a course if a formerly withdrawn student re-enrolls.
What time limitations can the support staff provide for restoring a single section? For example, if a course is erroneously deleted could it be restored to a state no older than 24 hours?
Ability to create archives in a locked/proprietary format.
Ability to create archives in an open format.
Hosting Priority  Score Comments
Complete cloud hosting service is available
Are the servers located in the US?
What is the redundancy plan?
Institutional Reporting Tools Priority  Score Comments
How does the LMS support course and program assessments?
Is there a range of reporting options in the LMS?
Does the LMS reporting allow drilling down to individual and program level activity?
Is add-on functionality a requirement for effective analysis of LMS data?
Migration Tool for Blackboard Vista Priority  Score Comments
Is a conversion from a .bak archive into the LMS easy and thorough?
How does your LMS migrate batches of sections from Blackboard Vista?
Does your system allow Common Cartridge export for reuse of content in other LMSs?
Describe a conversion into your LMS of a Vista 8.x course archive.
Security Priority  Score Comments
How does the solution ensure the highest degree of student privacy if needed, e.g. FERPA?
How does the solution protect confidential data and session activity both within the application and in transit?
How are data and session activity encrypted?
Detail how your solution supports mass notifications for users during emergencies, including any third party product integrations.
Describe in detail delivered security elements such as, but not limited to, the encryption of sensitive data, the enforcement of strong passwords, the enforcement of password changes, etc.?
SIS Integration Priority  Score Comments
Does your SIS API allow: automated class creation, enrollments, withdrawals, etc.? Detail additional API functionality.
How will UCF have access to back-end functions, database, and log files through the administrative interface or by other means?
List your primary SIS integration partners.
How does the LMS support SSL?
Describe in detail the robustness, real-time processing capabilities, and underlying technologies (e.g. web-based, file-based, etc.)
Browsers Compliance and JAVA Priority  Score Comments
The LMS supports Internet Explorer
The LMS supports Firefox
The LMS supports Chrome
The LMS supports Safari
The LMS supports Opera
JAVA is required to access multiple aspects of LMS.
Identify and detail any features not mentioned in this section that you wish to be considered.
Learning Context Hierarchy/Multi-Institution Priority  Score Comments
Ability to create a hierarchy
Ability to create multiple institutions
Ability to Brand various institutions
Ability to turn on/off system settings a various level in the hierarchy (Piloting a tool for a certain college)
Roles  Priority  Score Comments
Ability to assign privileges to system roles (various admin level)
Ability to assign privileges to course roles (various instructor levels)
Ability to assign admins to a selected group of courses (College of business admin)
Section 3. LEVEL OF PROFESSIONAL SERVICES REQUIRED:
The ability to deliver training to key administrative and academic end users is measured in this section.
Training and Client Support Priority  Score Comments
LMS will assign a client representative to work with the UCF account.
Training is offered for our support staff
Training would occur on UCF campus
Training materials and documentation are available that UCF can use to train faculty and students
Training material available at no extra cost
UCF would be allowed to modify the documentation to use our own “branding”
The LMS provides prerecorded online training materials
Section 4. THIRD PARTY INTEGRATIONS:
This section allows the applicant to detail their solution’s ability to integrate with key third party e-e-learning solutions and industry standards.
Integration Priority  Score Comments
Ability to integrate the LMS portal solution with PeopleSoft
Ability to import/export grades from the LMS using APIs
Ability to integrate publisher content via e-packs, cartridges, or packages with the LMS
Ability to integrate Respondus with the LMS
Ability to integrate Turnitin with the LMS
Ability to integrate our library systems with the LMS interface
Ability to integrate eInstruction with the LMS
Ability to integrate McGraw-Hill Course Connector and Connect with the LMS
Ability to integrate MyLab Suites with the LMS
Ability to share grades between integrated tool and the LMS.
Ability to integrate LockDown Browser with the LMS
Ability to integrate ThomsonNOW with the LMS?
Ability to integrate with student response systems (clickers)