-
Notifications
You must be signed in to change notification settings - Fork 0
Test Document Set Production Process
In order to develop the Search and Displace tool (the “Tool”), it will be necessary to have a body (the “Corpus”) of documents to test and adapt the tools functionality. The Corpus will comprise of 2 main elements and a related document. The first element will be raw documents. These documents will be unamended from original and will contain personal and potentially sensitive information (the “Sensitive Data”). The second element will be sanitised versions of the raw documents, where the Sensitive Data has been removed. The related document will be a tagging schema. When the Sensitive Data has been removed from the raw documents it will be replaced with a uniform and consistent set of tags. This will make testing easier, and may also assist in training/developing the Tool.
The raw documents can likely be from any source. Initially we will use legal contracts, but in later stages will move onto using other sources, like Wikipedia, blogs, etc. The reason for using legal documents initially is that one of originators of the project, Moorcrofts LLP understand them very well, they are well structured, and we understand the legal/licensing position of the documents. Another contributor, Contrai also have access to a reasonably large pool of them, so we can process them relatively quickly. We will explore the requirement for widening the types of raw documents as the project proceeds. We will need to research the legal position relating to the use of other document types. In some cases, using the documents for the purposes of developing the Tool may violate the licensing terms of those documents.
When we are processing the documents, we need to remove the target data. When we do this, we will need to leave a mark to indicate what has been removed. Removal, or sanitising of the data will only be one of the options available to users of the Tool, but in the development phase all data will be removed as part of the development of the tool. The schema will contain a set of tags (the “Tags”) which we will use to replace the data.
Whilst ultimately the goal of the Tool will be to identify particular data within a document, and then give the user options on what to do with those data, initially we will manually identify the data and replace it with the Tags. The Corpus will then allow us track the success of the tool in identifying the data.
As discussed, initially we will be using completed legal agreements. We have a bank of these available for use. They contain variety of data that should be removed/replaced (displaced!), so we believe they will be very useful for the initial stages of development. The logical structure of these documents may make it easier for the tool to identify the sensitive data. We do not believe this is necessarily an issue, but we will need to test and continue development with other document types. Subject to licensing restrictions we will gather other source material for the Tool to work on. Sources we are considering are Wikipedia, blogs, news sites, corporate sites. Realistically we can use any source material, as long as it has data that we want to remove.
The process is as follows: * Source material will be identified * Source material will be converted to our Raw Document format (exact format to be decided, will likely be a mark down format) * Sensitive Data will be manually identified and replaced with the Tags * Raw Documents and Sanitised Documents made available to the developers as a Corpus.
There will be three initial strands of work to develop the Corpus: (i) develop the tagging schema to use on the sanitised documents; (ii) sanitise the initial pool of legal documents; and (iii) identify additional source material to use in testing and development.