COMP90042_Project_Automatic_Fact_Verification

This is the project of COMP90042 Web Search and Analysis. Group work by 2 master students.

---

JSON Files Structure

"75397": {
	"claim": "Nikolaj Coster-Waldau worked with the Fox Broadcasting Company.",
	"label": "SUPPORTS",
	"evidence": [
		["Fox_Broadcasting_Company", 0],
		["Nikolaj_Coster-Waldau", 7]
	]
}

• claim: a fact that needs to verificate.
• label: show the result of this claim.
  • SUPPORTS : have evidence to support this claim.
  • REFUTES : have evidence to refute this claim.
  • NOT ENOUGH INFO : no evidence is provided for this claim.
• evidence: Show the evidence location
• "Fox_Broadcasting_Company" : show the page identifier.
• 0 : show the sentence index in that page.

---

Verification Process

>python score.py devset.json random-devset.json

deveset.json is the actual result.
random-devset.json is the prediction result.

In this project, we need edit the content of random-devset.json and increase the performance of the prediction system.

---

Report Requirements

• the description, analysis, and comparative assessment (where applicable) of methods used
• Ymention any choices you made in implementing your system along with empirical justification for those choices
• error analysis of the basic system to motivate your enhancements and describe it encough details
• evaluate whether your enhancements increased performance as compared to the basic system
• also report your relative performance on the codalab leaderboard
• (optional) discuss what steps you might take next if you were to continue development of your system

Ongoing Improvement

• Learning-to-Ranking: after retrieval top-K documents, use machine learning method to determine how many documents are the relevant evidence for this claim. It is better than just use the BM25 to select document.