Skip to content

Remove redundancy in MongoFields

David Sikter requested to merge remove-mongo-redundancy into v1.0.0

Removes redundant data storage in mongodb, and moves/renames some fields for consistency #588 (closed).

  • is renamed to Upload.upload_name, the field is removed from Calc.metadata to avoid redundancy, and upload_name is used consistently in the api
  • uploader, published and with_embargo fields are removed from Calc.metadata to avoid redundant storage in mongodb. The data is only stored on the upload level.
  • The field Upload.published is removed from mongodb, instead we check if Upload.publish_time is set. A property called published is created on Upload for convenience, so it is still possible to do things like upload.published (it's just not stored as a separate field in the DB).
  • Renamed Calc.parser to Calc.parser_name for consistency (affects mongodb and api)
  • Removed domain from Calc.metadata (the field is either fetched from the parser (if the parser has a domain set) or from the archive files, if set by the parser)
  • Removed processed from Calc.metadata, instead we fetch this value by looking at Calc.process_status
  • Removed license from Calc.metadata, and instead created a license-field on Upload.
  • Replacing the field upload_time with upload_create_time in EntryMetadata
  • Adding entry_create_time and publish_time to EntryMetadata.
  • Replacing Upload.create_time and Calc.create_time with Upload.upload_create_time and Calc.entry_create_time, for consistency

Merge request reports