Home » About the Center for Distributed Learning » Discover CDL » About Our Migration to Canvas » LMS Evaluation Checklist LMS Evaluation Checklist Assessments/Assignments/Grade BookPriority Score Comments Ability of LMS to allow browser lockdown techniques to ensure assessment security. Ability of LMS to allow proctor passwords to ensure assessment security. Ability of LMS to allow IP restrictions to ensure assessment can only be taken in certain locations (campus lab). Ability of testing tool to randomize quiz question answers. Ability of testing tool to randomize quiz questions. Ability to embed videos into assessments. Ability to except student Web pages and collaborative projects with multimedia. Ability to integrate course content with rubrics. Ability to send grades from LMS to Peoplesoft. Ability to set conditions (i.e., hide/show, dates) for multiple assessments at the same time. Ability to use rubrics to grade essay questions within an online assessment. Ability to copy rubrics from semester to semester Ability to use one rubric in multiple courses sections Assignment Submissions: PDF Assignment Submissions: Video Assignment Submissions: Web Page Assignment Submissions: Word Doc Assignment Submissions: Zip File Peer review tools are available. Quiz Question Type: Fill in the Blank/Short Answer Quiz Question Type: Long Answer/Paragraph Quiz Question Type: Matching Quiz Question Type: Multiple Choice Quiz Question Type: True or False Quiz Question Type: Word Scramble Ability to create graded pre and post-test surveys Group management features available for creating group assignments Ability to create a group discussion topic or assignment and then assign it to all groups at once Ability to provide detailed tracking of when students select an answer (i.e. time stamp of answer choice) within the Assessment tool Ability to re-grade assessments Ability to grade objective quizzes automatically and reveals results to students. Ability to selectively release assessments based on date. Ability to selectively release assessments based on time. Ability to selectively release assessments based on student ID. Ability to selectively release assessments based on grade book criteria. Ability to selectively release assessments based on multiple criteria. Ability for students to submit papers online and allow instructors to grade and give feedback without downloading the assignment. Ability to see student submission dates for an assignment in the grade book. Ability for students to see their grades (MyGrades Area). Ability to track to easily track attendance in the Grade Book (example: last login date). Ability to notify/alerts at-risk students. Ability to export and import from Excel into the Grade Book. Ability to include or support the use of portfolios to document student learning. Ability of system to support statistical analysis of assessment results. Self-assessments are available. Ability to change the file size limit on files that can be submitted to the Assignment Tool. Ability of testing tool to allow the use of math symbols. Ability of testing tool to allow the use of images. Ability of assessments be exported to another course. Ability to map learning outcomes to assessments in a way that achievement of specific outcomes can be evaluated. AnnouncementsPriority ScoreComments Ability for students to receive announcements outside the LMS for emergency notifications. Ability for students to view announcements within the LMS. Ability to copy announcements to new courses. Ability for System Admins can edit active (system) announcements. Ability for System Admins to edit/delete any role’s institution announcements. Ability for Instructors can edit active announcements. BlogsPriority ScoreComments LMS has a Blog tool. Ability to use the WYSIWIG or HTML editor within the Blog tool. BookmarksPriority ScoreComments Ability for instructors to bookmark areas within the course. Ability for students to bookmark areas within the course. CalendarPriority ScoreComments Ability to set start and stop time for calendar events. Ability to create a reoccurring calendar entries Ability to provide an RSS feed for students to subscribe Ability for users to subscribe to an external calendar Ability for calendar events to be added automatically from any tool that has a due date Copying ContentPriority ScoreComments Ability for instructors to easily copy (clone) from an existing section to a blank section. Ability for instructors to select specific items they would like to copy into a new course Ability to share content between courses. Ability to easily package a course to import into other LMS platforms or repositories. Chat and WhiteboardPriority ScoreComments LMS has a live chat feature. LMS has a live whiteboard feature. Ability to log/save chat transcripts. Ability of whiteboard to access/use an equation editor. Community NetworkingPriority ScoreComments Ability to use RSS to send notifications of updated content to students. Ability to include social media integration. Course TemplatesPriority ScoreComments Ability to create course templates Ability to assign ownership of course templates to a selected group of users Ability to assign course templates via the SIS integration Discussion ForumsPriority ScoreComments Threaded discussions are allowed. Ability to grade discussion postings Ability for students to attach documents in discussion forums. Ability for students to embed links in discussion forums. Ability to allow group creation in discussion forums. Ability to have anonymous postings. Ability to set release conditions (i.e., hide/show, dates) for multiple discussions at the same time. Ability to search Discussions Ability to moderate discussion forums Ability for students to have private discussions that cannot be viewed by the teacher. Ability to use rubrics to grade discussion forums. Ability to use the WYSIWIG or HTML editor in the discussion forums. IM EnabledPriority ScoreComments Ability for students and faculty see who is logged into courses. Ability for IM to permit file sharing. Ability for IM to permit two-way video. Built-in instant messaging capability. Ability for students and faculty to use IM to communicate. Ability for user to hide his/her presence in the IM tool. Internal E-mailPriority ScoreComments E-mail tool provides “spell checking” ability for narrative fields. Ability to set up rules within the e-mail tool. Ability to administratively set students’ LMS email address to a campus standard system for all students in all courses. Ability to forward internal e-mail to an external account. Ability to use the WYSIWIG or HTML editor in the internal e-mail tool. Ability to archive incoming and outgoing e-mails communications throughout the course. Ability to use search capabilities in the internal e-mail. Ability to create folders for filing/sorting e-mails within the e-mail tool. MashupsPriority ScoreComments Ability to enable learner and instructor mashups. MobilityPriority ScoreComments Ability to use mobile devices to access course content. All course tools are available in the mobile client Access to the LMS’s mobile client is standard. There is a pricing plan for mobile client access to the LMS. Available tools in the mobile client have the same functionality as the web version. The LMS is mobile-ready when using the web version through smart phones. The LMS is mobile-ready when using the web version through tablets. Online JournalPriority ScoreComments LMS have a journal tool. Ability to use the WYSIWIG or HTML editor in the journal tool. Online Marking Tools/NotesPriority ScoreComments Ability for students to make notations within the course. Search FunctionalityPriority ScoreComments Ability to use search within a course to find course content. SMS MessagingPriority ScoreComments Ability of LMS to send text message alerts. Social Media IntegrationPriority ScoreComments Ability to enable students to create social connections. Ability to integrate with Facebook. Ability to integrate with Twitter. Ability to integrate with YouTube. Student PortfoliosPriority ScoreComments Ability to access leading portfolio solutions through single sign-on LMS has a built-in portfolio solution Ability for students to access e-portfolios after graduation Ability to provide access to e-portfolios to students after graduation for free The portfolio remains active for a reasonable period of time What options does the user have to remain active after the period of time has expired? Ability to view discrete parts of the portfolio publicly What is the storage limitation per student and how flexible is the system for expansion? Student TrackingPriority ScoreComments Is detailed student system tracking available (log in, log out, last accessed, IP) Is detailed student tracking available for each course tool (How many discussion posts were read, how long did they spend on each quiz question) Can the tracking schedule be changed by System Admins Can tracking jobs be monitored from a GUI interface Third Party Test ManagementPriority ScoreComments Does your LMS support browser lockdown applications, like Respondus Lockdown Browser? Ability to import and export items in QTI (IMS Question and Test Interoperability). WikisPriority ScoreComments LMS has a Wiki tool. How does an instructor create wikis within the course? Describe the functionality of the wiki tool (e.g. versioning, HTML editor). WYSIWYG EditingPriority ScoreComments LMS has a built in fully functional WYSISYG or HTML editor that is available for students and faculty. LMS provides a fully functional equation editor. WYSISYG or HTML editor can modify HTML pages created outside of the LMS. The WYSIWIG allows you to apply formatting styles (e.g., heading styles, bulleted/numbered lists) Does your LMS provide a drag and drop, customizable environment (for example, utilizes technologies such as AJAX)? Can your LMS accommodate custom banners and icons on course homepage, header, footers, etc. Identify and detail any features not mentioned in this section that you wish to be considered. Section 2. FUNCTIONALITY OF LMS SOFTWARE FOR ADMINSTRATION This section covers the administrative interface and capabilities of the proposed system, as well as the level of compliance with interoperability and usability standards. AccessibilityPriorityScoreComments ADA/Section 508 compliance requirements are followed. Ability to support the use of screen readers (example: JAWS). Ability to set different assessment time frames for individual students (e.g., students with disabilities who need more time) Ability to provide a variety of feedback types (e.g., text, audio, video) for assignments, discussions, other student activities Ability to add audio/video components to quizzes or exams. Ability to add captions to assessments Authentication at Industry StandardsPriority ScoreComments Ability to support SSO from a PeopleSoft Portal. Ability to support authentication using LDAP. Ability to support authentication using Shibboleth. Ability to support authentication using Kerberos. Ability to support external and internal authentication. SSO SAML (Security Assertion Markup Language) is compliant. Branding FlexibilityPriority ScoreComments Ability to brand as Webcourses@UCF instance. Combine SectionsPriority ScoreComments Ability for faculty to combine section Ability for faculty to combine sections not listed under the same college Ability for System Admins to un-combine sections via the GUI Ability to combine sections via the SIS integration Concurrent UsersPriority ScoreComments Concurrent user capabilities Ability to scale the implementation for increased usage. Course cloning and archive restorationPriority ScoreComments Ability to handle backup and retrieval options at the system administrator level. Ability to use command line to create backups in batch. Ability to use GUI to create backups in batch. Ability for Blackboard Vista .bak archives to be restored in the LMS with student. Ability for Blackboard Vista .bak archives to be restored in the LMS without student. Ability to restore student artifacts in a course if a formerly withdrawn student re-enrolls. What time limitations can the support staff provide for restoring a single section? For example, if a course is erroneously deleted could it be restored to a state no older than 24 hours? Ability to create archives in a locked/proprietary format. Ability to create archives in an open format. HostingPriority ScoreComments Complete cloud hosting service is available Are the servers located in the US? What is the redundancy plan? Institutional Reporting ToolsPriority ScoreComments How does the LMS support course and program assessments? Is there a range of reporting options in the LMS? Does the LMS reporting allow drilling down to individual and program level activity? Is add-on functionality a requirement for effective analysis of LMS data? Migration Tool for Blackboard VistaPriority ScoreComments Is a conversion from a .bak archive into the LMS easy and thorough? How does your LMS migrate batches of sections from Blackboard Vista? Does your system allow Common Cartridge export for reuse of content in other LMSs? Describe a conversion into your LMS of a Vista 8.x course archive. SecurityPriority ScoreComments How does the solution ensure the highest degree of student privacy if needed, e.g. FERPA? How does the solution protect confidential data and session activity both within the application and in transit? How are data and session activity encrypted? Detail how your solution supports mass notifications for users during emergencies, including any third party product integrations. Describe in detail delivered security elements such as, but not limited to, the encryption of sensitive data, the enforcement of strong passwords, the enforcement of password changes, etc.? SIS IntegrationPriority ScoreComments Does your SIS API allow: automated class creation, enrollments, withdrawals, etc.? Detail additional API functionality. How will UCF have access to back-end functions, database, and log files through the administrative interface or by other means? List your primary SIS integration partners. How does the LMS support SSL? Describe in detail the robustness, real-time processing capabilities, and underlying technologies (e.g. web-based, file-based, etc.) Browsers Compliance and JAVAPriority ScoreComments The LMS supports Internet Explorer The LMS supports Firefox The LMS supports Chrome The LMS supports Safari The LMS supports Opera JAVA is required to access multiple aspects of LMS. Identify and detail any features not mentioned in this section that you wish to be considered. Learning Context Hierarchy/Multi-InstitutionPriority ScoreComments Ability to create a hierarchy Ability to create multiple institutions Ability to Brand various institutions Ability to turn on/off system settings a various level in the hierarchy (Piloting a tool for a certain college) Roles Priority ScoreComments Ability to assign privileges to system roles (various admin level) Ability to assign privileges to course roles (various instructor levels) Ability to assign admins to a selected group of courses (College of business admin) Section 3. LEVEL OF PROFESSIONAL SERVICES REQUIRED: The ability to deliver training to key administrative and academic end users is measured in this section. Training and Client SupportPriority ScoreComments LMS will assign a client representative to work with the UCF account. Training is offered for our support staff Training would occur on UCF campus Training materials and documentation are available that UCF can use to train faculty and students Training material available at no extra cost UCF would be allowed to modify the documentation to use our own “branding” The LMS provides prerecorded online training materials Section 4. THIRD PARTY INTEGRATIONS: This section allows the applicant to detail their solution’s ability to integrate with key third party e-e-learning solutions and industry standards. IntegrationPriority ScoreComments Ability to integrate the LMS portal solution with PeopleSoft Ability to import/export grades from the LMS using APIs Ability to integrate publisher content via e-packs, cartridges, or packages with the LMS Ability to integrate Respondus with the LMS Ability to integrate Turnitin with the LMS Ability to integrate our library systems with the LMS interface Ability to integrate eInstruction with the LMS Ability to integrate McGraw-Hill Course Connector and Connect with the LMS Ability to integrate MyLab Suites with the LMS Ability to share grades between integrated tool and the LMS. Ability to integrate LockDown Browser with the LMS Ability to integrate ThomsonNOW with the LMS? Ability to integrate with student response systems (clickers)
