Skip to Main Content

LMS Evaluation Planning Toolkit: Implement

This toolkit houses material created and identified by PALNI's LMS Evaluation Task Force to assist member institutions embarking upon the process of an existing Learning Management System (LMS) examination and/or change.

Overview

The implement phase is where each stage of the process comes together. There isn't a cookie cutter approach to implementing a new LMS system, though suggestions are offered in the tabs below with each tab being a specific step in the implementation timeline.  Keep in mind that this process can also work when major system upgrades are deployed for your current system.

Implementation Considerations

implementation team image

 

Implementation Team

 

As you begin the implementation phase, it is necessary to assemble your core implementation team who will be directly responsible with implementation tasks and making decisions along the way.

  • Size of the core team is reflective of the size of your institution. (2-3 for smaller institutions; 5-6 for larger institutions)
  • Role of the core team members:
    • Team Leader - responsible for keeping timeline tasks moving forward, working with team members to remove obstacles & plan contingencies, and interact with the vendor directly
    • Project Manager - responsible for keeping track of all tasks, resources and due dates. 
    • eLearning Technologist - responsible for representing the institution's eLearning tools, platforms, and instructional models. Primary tasks relate to managing course migration and testing.
    • User Support Trainer - responsible for institution user support and administrative training needs, as well as configuration of course functionalities, user profiles, assignments & notifications, reports, etc.
    • IT Specialist - responsible for system tasks including hosting and installation, security, user account management & authentication, data migration, system integrations, etc.
  • Role of an extended team of frequent/early users to be utilized at key points in the implementation process. Consider the following extended users:
    • instructional designers
    • curriculum administrators
    • registrars
    • additional IT specialists (production support, database administrators, system integrators, network managers, security officers, etc)

implementation plan image

 

Implementation Plan

 

The context of this entire toolkit surrounds the topic of planning, but it's important to note that planning should be be done throughout the various stages of the process. During the implementation phase, unanticipated needs and challenges will arise, leaving room for readjustment and further expansion of your implementation plan. This is where your Project Manager can take the lead in assisting with the re-planning tasks.

Plan specifics will vary depending on your institutional needs. Consider these suggestions to assist you in the planning process:

  • Ask the LMS vendor to provide a project-plan template (keeping in mind this will be from the vendor's point of view)
  • Include tasks such as:
    • data cleanup
    • communications planning
    • change management planning
    • user acceptance testing
    • user support planning

 

configuration image

 

Configuration Considerations

 

Up-front configuration decisions can be difficult when still learning a new LMS system, but vendors will expect you to make system configuration decisions based on your knowledge of your own institution and what you've learned about the system itself. It's important to think through various configuration settings as making changes in the future may prove to be difficult or impractical. Keep in mind, it's not the vendors responsibility to know the ins and outs of each of their customer's organizations, so utilizing your Core Team will be beneficial to arriving at the best possible outcome.

Configuration considerations will include, but may not be limited to:

  • an understanding of your data and operations
  • an understanding of the system's data fields, functionality, and capabilities

User Profiles
Profile data will help you segment user permissions and may include user name, contact info, grade/year, degree program, major, transfer credits, etc.

  • What user demographic data is needed?
  • Where is the data currently stored/managed?

Administrator Security
Security roles are a set of permissions assigned to each type of LMS administrator (i.e. 1 person to run reports, 1 group to create/schedule courses, etc.). The process includes:

  • establishing security roles
  • configuring the appropriate security role permissions
  • assigning users to the respective roles

Course/Curriculum Structures
A course or curriculum structure include sequential user learning activities where some are mandatory, and others are optional. This stage of configuration must be completed once course data has been fully migrated. Prior to migration, administrators can:

  • experiment with placeholder courses to establish models and templates
  • observe LMS behavior with each test configuration
  • make adjustments to configuration
  • replicate the structure post-migration

Evaluations & Assessments
Using evaluations and assessments within the LMS will allow users to easily view reports at the question level, enabling you to perform item analysis and determine validity of your survey or test. When switching LMS products consider:

  • assessment data may need to be redeveloped
  • 3rd party survey tools will need to be tested to determine behavior with the new LMS (review tool vendor and LMS vendor support)

Notifications
Some LMS systems allow custom notifications in addition to the default notification settings. System administrators will need to decide which notifications to activate and deactivate. Settings include:

  • alerts 
  • reminders
  • confirmations

A good rule of thumb is to turn off all notifications that are not essential in order to avoid "spamming" users with too many notifications. Additionally, administrators will want to test how the LMS email function "talks with" the institution's email tool. 

Reports
As with notifications, some LMS systems allow for customized reports. It's important to define your report requirements early in the implementation process, keeping in mind:

  • all necessary reports needed, and for whom
  • how often reports should be run, and for what purpose, and if auto-run capabilities are present
  • what set of data and format is needed when exporting

Knowing this information up front will help guide many LMS configuration and data migration decisions. 

system integrations image

 

System Integrations

 

An LMS may integrate with a number of systems described below:

User Account & Profile Systems
Institutions likely have a student information system (SIS) in place, utilized by the registrar. For enrollment purposes, the LMS will need to receive information from the SIS on an automatic or manually run process.

Single Sign-On (SSO)
In order to avoid users being required to log into different systems with different logins, institutions can implement single sign-on. This will enable users to log into the network once and gain access to multiple systems through SSO authentication. Your IT support representative will need to discuss options with the LMS vendor to determine how to best implement SSO in your organization.

Portal
LMS systems generally offer two methods of integration with a portal:

  • Deep Linking: enables you to capture the web address of a specific course in your LMS and paste it into a portal page.
  • API: This application programming interface allows your IT department to access the data and functionality of your LMS to be posted in the portal.

file migration image

 

Course & Data Migration Considerations

 

Migration to the new LMS system is one of the more complex tasks in the implementation phase as you complete migration in a specific sequence and must address any incompatibilities along the way. It's also recommended to migrate in stages:

  1. Begin with a small sample of data and test to verify the migration is working properly
  2. Migrate all data to date
  3. Enhance course setups by configuring any features and functionality that you may not have had in your previous system

To determine what and how much to migrate, consider the following:

Data Retention Policies

  • Begin by migrating as little as possible. The more data you migrate, the greater risk of potential incompatibility errors which may delay your implementation timeline.
  • Know your institution's policies for the length of time user records are retained.
  • Whatever information is not migrated, opt to archive older data. IT staff can backup and restore any archived data when/if needed.

User Data

  • Load users into the new LMS
    • Prior to the load, ensure that this process has been developed and tested
    • Determine if you are loading user profiles with an established feed from another system
    • Determine if you are loading user profiles directly from your old LMS system

Standards-based Courseware Migration
When migrating courses, you will need to export your courseware in the appropriate format of the new LMS before importing. Migrating with standards will make the process easier:

  • IMS Common Cartridge (CC)
  • IMS Learning Tools Interoperability (LTI)
  • IMS Question and Test Interoperability (QTI)

Branding
Prior to testing and live production of the new system, consider any branding needs that should be addressed. Create branded materials at this time such as customized landing page templates, etc.


Items to Note:

  • Your new system support package may include different levels of migration support. Depending on the level of support your institution purchased, the LMS vendor may perform the entire migration for you - including file cleanup.
  • If your package does not include full migration support, you will need to provide a map of files to be migrated. The vendor will likely fine tune and clean up the files prior to placing them in the new system.

user testing image

 

User Testing


 

In this stage of the process, you want to be as thorough as possible to test every part of the system. Identifying bugs, kinks, and configurations that may need to be altered is crucial prior to going live.

Administrators

  • Begin with the core team members brainstorming a list of procedures and tasks that will be routinely run. Divide up the list among the members to begin...
    • running each procedure
    • making note in a collaborative spreadsheet of every action performed (menus selected, fields entered, checkboxes checked, etc. (include a description of the action, the expected result, and the actual result to help identify problem areas)
  • Those configuring and supporting the system should test to make sure configurations, courses, and data are available in the system as you expect them to be.

Users

  • During this phase, users will test all system functionalities to ensure the vendor has delivered a fully working, bug-free system.
  • Users testing the system should include instructors, students, and any relevant staff
  • Gather feedback 
     

user testing and training image

 

Training

 

Offering group trainings, as well as 1:1 user support will be vital to ensuring users are adequately prepared and equipped. When planning these trainings, consider the following:

  • Mode of Training: workshop vs. individual consultations
  • Audience: faculty, students, staff
  • Support Documentation: institutionally-created videos, FAQ systems, handouts vs. vendor-created documentation
  • Features to Highlight: 
    • General Workshop: system interface, user profile, course settings, course content, notifications
    • Advanced Workshop: gradebook, group work, 3rd party integrations

 

go live image

 

Go Live: Production

 

As you move into the production phase, consider the following:

Helpdesk Support
User support will be steady during the "Go Live" phase of implementation. Have the following in mind as you move forward:

  • Support Staff: Know who will be directly involved in offering larger user training and 1:1 user support
    • Will you offer walk-in workshops for 1:1 support?
    • Will you offer formal training sessions for larger groups?
  • Anticipate Questions: Create a list of anticipated topics and a shared knowledge of how support staff will respond
    • how to connect to the LMS
    • recovering lost passwords
    • how to publish courses
    • adding & removing users
    • adding & removing course from your individual LMS menu
    • adding & removing LMS tools
    • uploading your course syllabus
    • setting up the grade book

Blackout Period
There will likely be a period of time when neither your old system, nor the new LMS will be available. During this time, administrators will want to:

  • shut down the old system
  • migrate any final data
  • redirect the web address access points to the new LMS system

Literature Review

Gautreau, C. (2011). Motivational factors affecting the integration of a learning management system by facultyJournal of Educators Online, 8(1), 1-25. 

Morgan, G. (2003). Faculty use of course management systems, volume 2. Boulder, CO: Educause Center for Applied Research. Retrieved from https://www.educause.edu/ir/library/pdf/ers0302/rs/ers0302w.pdf

 

 

 

Change Management & Communication Tips

Communication Tips

Marketing the new system is the most important communication step during the implementation phase. Consider the following:

  • Methods: 
    • website devoted to the system
    • digital signs around campus
    • email messages to various audiences
  • Initiator:
    • who will share the information - academic office, center for teaching & learning, IT department, core system support team?
  • Mode:
    • ​faculty governance meetings
    • departmental meetings
    • listservs

Items to keep in mind:

  • Archives of course content in the old system may not be complete. When exporting course content, student data (submissions, grades, etc.) likely will not be included in the migration. When communicating system access timelines to your faculty, include this warning (several times, and in both written and verbal form) that they may in fact lose access to student submissions, outcome artifacts, and gradebooks. 
  • If faculty need the above information for personal or accreditation purposes, consider training material which details how to downloaded student data.