From: Thierry Reding treding@nvidia.com Subject: scripts/spdxcheck.py: always open files in binary mode
The spdxcheck script currently falls over when confronted with a binary file (such as Documentation/logo.gif). To avoid that, always open files in binary mode and decode line-by-line, ignoring encoding errors.
One tricky case is when piping data into the script and reading it from standard input. By default, standard input will be opened in text mode, so we need to reopen it in binary mode.
The breakage only happens with python3 and results in a UnicodeDecodeError (according to Uwe).
Link: http://lkml.kernel.org/r/20181212131210.28024-1-thierry.reding@gmail.com Fixes: 6f4d29df66ac ("scripts/spdxcheck.py: make python3 compliant") Signed-off-by: Thierry Reding treding@nvidia.com Reviewed-by: Jeremy Cline jcline@redhat.com Cc: Thomas Gleixner tglx@linutronix.de Cc: Jonathan Corbet corbet@lwn.net Cc: Joe Perches joe@perches.com Cc: Uwe Kleine-König u.kleine-koenig@pengutronix.de Cc: stable@vger.kernel.org Signed-off-by: Andrew Morton akpm@linux-foundation.org ---
scripts/spdxcheck.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
--- a/scripts/spdxcheck.py~scripts-spdxcheckpy-always-open-files-in-binary-mode +++ a/scripts/spdxcheck.py @@ -168,6 +168,7 @@ class id_parser(object): self.curline = 0 try: for line in fd: + line = line.decode(locale.getpreferredencoding(False), errors='ignore') self.curline += 1 if self.curline > maxlines: break @@ -249,12 +250,13 @@ if __name__ == '__main__':
try: if len(args.path) and args.path[0] == '-': - parser.parse_lines(sys.stdin, args.maxlines, '-') + stdin = os.fdopen(sys.stdin.fileno(), 'rb') + parser.parse_lines(stdin, args.maxlines, '-') else: if args.path: for p in args.path: if os.path.isfile(p): - parser.parse_lines(open(p), args.maxlines, p) + parser.parse_lines(open(p, 'rb'), args.maxlines, p) elif os.path.isdir(p): scan_git_subtree(repo.head.reference.commit.tree, p) else: _