databricks pyspark -blobstoragerotatingfilehandler log到单个文件中的azure blob容器
我正在使用BlobStoragerotatingFileHandler将日志从数据链球上写入Azure Blob容器。我的代码有3个日志语句，我希望将所有3个写入一个文件，但是每个语句都写入BLOB容器中的单独文件。如何在一个文件中获取所有日志语句？
import logging
from azure_storage_logging.handlers import BlobStorageRotatingFileHandler
log = logging.getLogger('service_logger')
log.setLevel(logging.DEBUG)
azure_blob_handler = BlobStorageRotatingFileHandler(filename = "name.csv",account_name="xyz",account_key="&&&",maxBytes=5000,container="abc")
azure_blob_handler.setLevel(logging.DEBUG)
log.addHandler(azure_blob_handler)
log.info("Start processing file....")
log.info("Processsed file successfully")
log.info("End file successfully")
I am using BlobStorageRotatingFileHandler to write logs from databricks to an Azure blob container. My code has 3 log statements and I want all 3 to be written to a single file, but each statement is being written to a separate file in the blob container. How do I get all log statements in one file?
import logging
from azure_storage_logging.handlers import BlobStorageRotatingFileHandler
log = logging.getLogger('service_logger')
log.setLevel(logging.DEBUG)
azure_blob_handler = BlobStorageRotatingFileHandler(filename = "name.csv",account_name="xyz",account_key="&&&",maxBytes=5000,container="abc")
azure_blob_handler.setLevel(logging.DEBUG)
log.addHandler(azure_blob_handler)
log.info("Start processing file....")
log.info("Processsed file successfully")
log.info("End file successfully")
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
根据这一点>直到日志内容未达到maxbytes并在超过maxbytes时继续上传多个文件之前，请勿上传日志。
。以某种方式附加了时间戳（即，日志上载的时间）。
您可以尝试的解决方法之一就是使用
blobStoragerotatingFileHandler您可以尝试使用tableStorageHandler，其中结果以下面的格式显示。REFERENCES: < a href="https://pypi.org/project/azure-storage-logging/#:%7E:text=The%20BlobStorageRotatingFileHandler%20class%20is%20a%20subclass%20of%20logging.handlers.RotatingFileHandler,when%20the ％20元％20FILE％20REACHES％20A％20 caution％20尺。” rel =“ nofollow noreferrer”> azure-nordoraging-loging
According to this document,
RotatingFileHandlerdoes not upload the log until the log content does not reach the maxBytes and proceeds to upload multiple files consequently when it exceeds the maxBytes.Another reason you are facing this issue is because the filename of the blob is somehow getting appended with the timestamp (i.e., the time at which the log is getting uploaded).
One of the workarounds that you can try is instead of using
BlobStorageRotatingFileHandleryou can try usingTableStorageHandlerwhere the results are displayed in the below format.REFERENCES: azure-storage-logging