Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
On Thursday, 7th July from 1 to 3 pm there will be a maintenance with a short downtime of GitLab.
Open sidebar
nomad-lab
nomad-FAIR
Commits
a9f1a4e8
Commit
a9f1a4e8
authored
Oct 17, 2019
by
Alvin Noe Ladines
Browse files
Added xz compression
parent
a1c76ed1
Pipeline
#62088
failed with stages
in 31 minutes and 58 seconds
Changes
2
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
nomad/parsing/__init__.py
View file @
a9f1a4e8
...
...
@@ -71,7 +71,7 @@ based on NOMAD-coe's *python-common* module.
:members:
"""
from
typing
import
Callable
,
IO
,
Union
from
typing
import
Callable
,
IO
,
Union
,
Dict
import
magic
import
gzip
import
bz2
...
...
@@ -118,7 +118,7 @@ def match_parser(mainfile: str, upload_files: Union[str, files.StagingUploadFile
with
open
(
mainfile_path
,
'rb'
)
as
f
:
compression
,
open_compressed
=
_compressions
.
get
(
f
.
read
(
3
),
(
None
,
open
))
with
open_compressed
(
mainfile_path
,
'rb'
)
as
cf
:
with
open_compressed
(
mainfile_path
,
'rb'
)
as
cf
:
# type: ignore
buffer
=
cf
.
read
(
config
.
parser_matching_size
)
mime_type
=
magic
.
from_buffer
(
buffer
,
mime
=
True
)
...
...
nomad/parsing/parser.py
View file @
a9f1a4e8
...
...
@@ -139,8 +139,7 @@ class MatchingParser(Parser):
if
self
.
_mainfile_contents_re
is
not
None
:
try
:
# Try to open the file as a string for regex matching.
decoded_buffer
=
buffer
.
decode
(
'utf-8'
)
print
(
'DECODED'
,
decoded_buffer
)
except
UnicodeDecodeError
as
e
:
except
UnicodeDecodeError
:
return
False
# We're looking for a string match in a file that can't be converted to string.
if
self
.
_mainfile_contents_re
.
search
(
decoded_buffer
)
is
None
:
return
False
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment