Limitations

This wiki page discusses the limitations of the LMSDataHarvester, why they exist and in some cases, how they can be removed.

#Destination database must be initialized manually Currently the destination database must be manually initialized with its schema prior to using it. It should be possible to add support for automatically initializing a blank destination database, as JPA supports the operation, however it is not currently implemented.

#Does not handle all possible activities There are a potentially infinite number of activities. As such, at any given time only a subset of all of the activities will be supported. See Supported Activities for the list of currently supported activities and Configuring Activities for instructions on how to add new activities.

#Activity content is not sanitized Activity content may contain user identifying information. Scrubbing the activity data to remove user identifying information is dependent on the activity implementation, since the data would be different for each activity. Currently, none of the supported activities has support for detecting and removing user identifying information.

#Enrolments are not extracted from Moodle Enrolments and roles have different meanings for Moodle and the research (destination) database. In the research database an enrolment indicates that a user has participated in a course, with the role indicating the nature of that participation. As a result, enrolments are permanent and may only have one role. Moodle uses an enrolment to indicate that a user is currently participating in a course and the associated roles provide access permissions. Enrolments in Moodle can be deleted at any time, and may have multiple roles. Furthermore, the research database includes the final grade, and an indication if the user has given consent for their data to be used for research. Both of these pieces of information are not present in Moodle. Due to the differences in the interpretation of enrolments and roles the enrolment information can not be reliably extracted from Moodle.

#Grades are not extracted from Moodle Currently, extracting grades from Moodle is not implemented. To implement grade extraction from Moodle, a class must be developed (and mapped to the Moodle database) which can load entries from the Moodle grade book. The grade book entries would then need to be converted to the format of the research database.

#Only one course can be processed per invocation Processing the data for one course uses a lot of memory. Under some conditions it is possible to exhaust the Java Virtual Machine's heap. Attempting to process the data for multiple courses at a time would increase the memory consumption. Due to the memory consumption while processing the data for a course and the JPA performance issues with large numbers of records the Harvester is restricted to processing one course per invocation.

#Can not update existing courses Currently it is not possible to use the Harvester to add data to a course which already exists in the destination database. The element classes Activity, Enrolment, LogEntry and SubActivity do not have instances which are unique within the domain model. As a result it is not currently possible to construct a mapping between an instance of one of these classes in the destination database to the corresponding instance in the source database. Attempting to perform an update without having the instances of these classes mapped to the corresponding instances in the source database would result in all of the instances of these classes in the destination database being duplicated. It may be possible to infer the mappings between the instances in the source and destination databases.

The mappings between the non-unique element instances in the destination database and the corresponding instances in the source database can be reconstructed. If the user data is available, then each enrolment instance may be uniquely identified since a user may only be enrolled in a given course once. LogEntry instances include a time-stamp (with second precision). As a result most LogEntry instances will be unique. For the remaining LogEntry instances the relative ordering can be used to construct a mapping since the relative ordering from the source database is preserved in the destination database. Finally, the mappings for the Activity, and SubActivity instances can be deduced from their relationships with the LogEntry instances.