0

I am writing some logs in a file using logging module

#filename : demo.py
import logging
#other imports as well
logging.basicConfig(filename="myfile.log",
                    format='%(asctime)s %(funcName)s %(levelname)s %(message)s',
                    filemode='w')
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
logger.debug("Writing logs in this way")

But sometimes my module writes lots of nullvalues

^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@

So I have written a script to remove these values. But when I modify myfile.log my further logs are not stored. I know that this is due to change/close of file stream of myfile.log. Is it possible to modify my log file without hurting or losing my already existing stream which demo.py has. I am using ubuntu : 18.04 and python : 3.6.9
For modification I tried the following ways

  1. I am using script sed -i -e 's/\o00//g' myfile.log to modify file
  2. I manually edit file using vim : vim myfile.log To delete line I used dd then closed using :wq

From a comment https://stackoverflow.com/questions/64875452/modify-a-file-without-changing-its-file-stream I know that both method over write new file over old file. Is there a way to edit file inplace.

Pranjal Doshi
  • 101
  • I don't know Python, but I can connect the dots. You're using filemode='w' instead of a. It's like > vs >> in this answer of mine. If something truncates the file then your logger will make the file "regain its old size and grow further, missing data will be filled with zeros (in a sparse way, if possible)". What process truncates the file? I don't know, logrotate maybe. Also see how this is flawed in general. – Kamil Maciorowski Nov 17 '20 at 15:29
  • This also means that if you manage to remove these null bytes in place and you will reduce the size of the file, your logger will keep its ever-growing offset inside the file and null bytes will soon appear again. If I'm right, then what you want to do is futile because of filemode='w'. – Kamil Maciorowski Nov 17 '20 at 15:36
  • @KamilMaciorowski so if I use filemode='a' then it should solve the issue.. right? – Pranjal Doshi Nov 17 '20 at 15:40
  • Probably. Still (from the linked article): "having the log-rotater program separate from the log-writer program results in several unavoidable potential problems". – Kamil Maciorowski Nov 17 '20 at 15:45
  • @KamilMaciorowski Thanks for your help :) – Pranjal Doshi Nov 17 '20 at 15:53

0 Answers0