This is the README for the mention level dataset for 'Identifying civilians killed by police with distantly supervised entity-event extraction' presented at EMNLP 2017.

Files
=======
There are two files: 
- train.json contains 132,833 mentions of possible police fatality victims from Jan. 1, 2016 to Aug. 31, 2016.
- test.json contains 68,925 mentions of possible police fatality victims from Sept. 1, 2016 to Dec. 31, 2016.   
- Both files are saved in .json format with one document per line as a dictionary and dictionary keys:
  - "docid" : docid_sentid_fragid, i.e. the first number before the hyphen is the docid matching PoliceKillingsExtraction-html-v1/ the second number is the sentence number within that document (sentence segementation in preproc) and the third number is the fragment id 
  - "downloadtime": date the scraper downloaded the document
  - "name": HAPNIS normalized name for that particular mention that will be mapped to "TARGET" symbol
  - "names_org": original names in the text that map to "name"
  - "sentnames": other names in the mention that get mapped to "PERSON" symbol
  - "sent_org": the raw mention text
  - "sent_alter": the mention text with names replaced with"TARGET" and "PERSON" symbols 
  - "plabel": is 1 if "name" matches a HAPNIS normalized Fatal Encounters name, 0 otherwise

Citations
====
If you make use of our data please cite: 

@inproceedings{keith2017,
	Author = {Katherine A. Keith and Abram Handler and Michael Pinkham and Cara Magliozzi and Joshua McDuffie and Brendan O'Connor},
	Booktitle = {Proceedings Empirical Methods in Natural Language Processing},
	Publisher = {Association for Computational Linguistics},
	Title = {Identifying civilians killed by police with distantly supervised entity-event extraction.},
	Year = {2017},
	}