LMS Evaluation Planning Toolkit: Implement

Implementation Plan

The context of this entire toolkit surrounds the topic of planning, but it's important to note that planning should be be done throughout the various stages of the process. During the implementation phase, unanticipated needs and challenges will arise, leaving room for readjustment and further expansion of your implementation plan. This is where your Project Manager can take the lead in assisting with the re-planning tasks.

Plan specifics will vary depending on your institutional needs. Consider these suggestions to assist you in the planning process:

• Ask the LMS vendor to provide a project-plan template (keeping in mind this will be from the vendor's point of view)
• Include tasks such as:
  • data cleanup
  • communications planning
  • change management planning
  • user acceptance testing
  • user support planning

Configuration Considerations

Up-front configuration decisions can be difficult when still learning a new LMS system, but vendors will expect you to make system configuration decisions based on your knowledge of your own institution and what you've learned about the system itself. It's important to think through various configuration settings as making changes in the future may prove to be difficult or impractical. Keep in mind, it's not the vendors responsibility to know the ins and outs of each of their customer's organizations, so utilizing your Core Team will be beneficial to arriving at the best possible outcome.

Configuration considerations will include, but may not be limited to:

• an understanding of your data and operations
• an understanding of the system's data fields, functionality, and capabilities

User Profiles
Profile data will help you segment user permissions and may include user name, contact info, grade/year, degree program, major, transfer credits, etc.

• What user demographic data is needed?
• Where is the data currently stored/managed?

Administrator Security
Security roles are a set of permissions assigned to each type of LMS administrator (i.e. 1 person to run reports, 1 group to create/schedule courses, etc.). The process includes:

• establishing security roles
• configuring the appropriate security role permissions
• assigning users to the respective roles

Course/Curriculum Structures
A course or curriculum structure include sequential user learning activities where some are mandatory, and others are optional. This stage of configuration must be completed once course data has been fully migrated. Prior to migration, administrators can:

• experiment with placeholder courses to establish models and templates
• observe LMS behavior with each test configuration
• make adjustments to configuration
• replicate the structure post-migration

Evaluations & Assessments
Using evaluations and assessments within the LMS will allow users to easily view reports at the question level, enabling you to perform item analysis and determine validity of your survey or test. When switching LMS products consider:

• assessment data may need to be redeveloped
• 3rd party survey tools will need to be tested to determine behavior with the new LMS (review tool vendor and LMS vendor support)

Notifications
Some LMS systems allow custom notifications in addition to the default notification settings. System administrators will need to decide which notifications to activate and deactivate. Settings include:

• alerts 
• reminders
• confirmations

A good rule of thumb is to turn off all notifications that are not essential in order to avoid "spamming" users with too many notifications. Additionally, administrators will want to test how the LMS email function "talks with" the institution's email tool. 

Reports
As with notifications, some LMS systems allow for customized reports. It's important to define your report requirements early in the implementation process, keeping in mind:

• all necessary reports needed, and for whom
• how often reports should be run, and for what purpose, and if auto-run capabilities are present
• what set of data and format is needed when exporting

Knowing this information up front will help guide many LMS configuration and data migration decisions. 

System Integrations

An LMS may integrate with a number of systems described below:

User Account & Profile Systems
Institutions likely have a student information system (SIS) in place, utilized by the registrar. For enrollment purposes, the LMS will need to receive information from the SIS on an automatic or manually run process.

Single Sign-On (SSO)
In order to avoid users being required to log into different systems with different logins, institutions can implement single sign-on. This will enable users to log into the network once and gain access to multiple systems through SSO authentication. Your IT support representative will need to discuss options with the LMS vendor to determine how to best implement SSO in your organization.

Portal
LMS systems generally offer two methods of integration with a portal:

• Deep Linking: enables you to capture the web address of a specific course in your LMS and paste it into a portal page.
• API: This application programming interface allows your IT department to access the data and functionality of your LMS to be posted in the portal.

Course & Data Migration Considerations

Migration to the new LMS system is one of the more complex tasks in the implementation phase as you complete migration in a specific sequence and must address any incompatibilities along the way. It's also recommended to migrate in stages:

1. Begin with a small sample of data and test to verify the migration is working properly
2. Migrate all data to date
3. Enhance course setups by configuring any features and functionality that you may not have had in your previous system

To determine what and how much to migrate, consider the following:

Data Retention Policies

• Begin by migrating as little as possible. The more data you migrate, the greater risk of potential incompatibility errors which may delay your implementation timeline.
• Know your institution's policies for the length of time user records are retained.
• Whatever information is not migrated, opt to archive older data. IT staff can backup and restore any archived data when/if needed.

User Data

• Load users into the new LMS
  • Prior to the load, ensure that this process has been developed and tested
  • Determine if you are loading user profiles with an established feed from another system
  • Determine if you are loading user profiles directly from your old LMS system

Standards-based Courseware Migration
When migrating courses, you will need to export your courseware in the appropriate format of the new LMS before importing. Migrating with standards will make the process easier:

• IMS Common Cartridge (CC)
• IMS Learning Tools Interoperability (LTI)
• IMS Question and Test Interoperability (QTI)

Branding
Prior to testing and live production of the new system, consider any branding needs that should be addressed. Create branded materials at this time such as customized landing page templates, etc.

Items to Note:

• Your new system support package may include different levels of migration support. Depending on the level of support your institution purchased, the LMS vendor may perform the entire migration for you - including file cleanup.
• If your package does not include full migration support, you will need to provide a map of files to be migrated. The vendor will likely fine tune and clean up the files prior to placing them in the new system.

User Testing

In this stage of the process, you want to be as thorough as possible to test every part of the system. Identifying bugs, kinks, and configurations that may need to be altered is crucial prior to going live.

Administrators

• Begin with the core team members brainstorming a list of procedures and tasks that will be routinely run. Divide up the list among the members to begin...
  • running each procedure
  • making note in a collaborative spreadsheet of every action performed (menus selected, fields entered, checkboxes checked, etc. (include a description of the action, the expected result, and the actual result to help identify problem areas)
• Those configuring and supporting the system should test to make sure configurations, courses, and data are available in the system as you expect them to be.

Users

• During this phase, users will test all system functionalities to ensure the vendor has delivered a fully working, bug-free system.
• Users testing the system should include instructors, students, and any relevant staff
• Gather feedback 
   

Training

Offering group trainings, as well as 1:1 user support will be vital to ensuring users are adequately prepared and equipped. When planning these trainings, consider the following:

• Mode of Training: workshop vs. individual consultations
• Audience: faculty, students, staff
• Support Documentation: institutionally-created videos, FAQ systems, handouts vs. vendor-created documentation
• Features to Highlight: 
  • General Workshop: system interface, user profile, course settings, course content, notifications
  • Advanced Workshop: gradebook, group work, 3rd party integrations

Go Live: Production

As you move into the production phase, consider the following:

Helpdesk Support
User support will be steady during the "Go Live" phase of implementation. Have the following in mind as you move forward:

• Support Staff: Know who will be directly involved in offering larger user training and 1:1 user support
  • Will you offer walk-in workshops for 1:1 support?
  • Will you offer formal training sessions for larger groups?
• Anticipate Questions: Create a list of anticipated topics and a shared knowledge of how support staff will respond
  • how to connect to the LMS
  • recovering lost passwords
  • how to publish courses
  • adding & removing users
  • adding & removing course from your individual LMS menu
  • adding & removing LMS tools
  • uploading your course syllabus
  • setting up the grade book

Blackout Period
There will likely be a period of time when neither your old system, nor the new LMS will be available. During this time, administrators will want to:

• shut down the old system
• migrate any final data
• redirect the web address access points to the new LMS system

Literature Review

Gautreau, C. (2011). Motivational factors affecting the integration of a learning management system by faculty. Journal of Educators Online, 8(1), 1-25. 

Morgan, G. (2003). Faculty use of course management systems, volume 2. Boulder, CO: Educause Center for Applied Research. Retrieved from https://www.educause.edu/ir/library/pdf/ers0302/rs/ers0302w.pdf.