[Prev] Thread [Next]  |  [Prev] Date [Next]

Logger for multiprocesses into one "master"-process? Palic, Darko Fri Dec 17 07:00:55 2010

Hello developers,

first of all, many thanks for the log framework which works like we are used under Java. It runs like a charm!

I am not a native speaker, so foregive me if I miss the spelling or the grammar.

Currently I need a logger in a multiprocess-environment which need a log-sink in one master-process. To be more specific: I have at least 20 processes (not threads) which need logging. One process, the master process must collect all logging events of him self and the other 20 processes.

So I checked if we could you the ODBCAppender. This does not fit our needs, due to the fact, that the DB will not always be available. The SocketAppender is a similar thing, the appender needs a open Socket to communicate. Additionally I couldn't find a Socket"Sink" there the serialized LogEvents can be read from the Socket and put into an other Appender like the RollingFileAppender.
And here is my problem and also a wish to enhance log4cxx.

1. What is the usage of the SocketAppender? Must I use Chainsaw to get the serialized LogEvents, or is it also possible to collect these LogEvents in an SocketReader, which itself can easily pass the deserialized LogEvents to an other Appender?
App1 => write LogEvent via SocketAppender => SocketTransmission => App2 => read/deserialize LogEvent => pass LogEvent into App2-Appender

2. Is the SocketAppender able to pass the NestedDiagnosticContext so the deserialized LogEvent can use the previous NDC for logging? I do not know the implementation details of the LogEvents and also the NDC

3. The socket for the interprocess logging is too unefficient for our log-usage. So I've implemented a message_queue, which is used as the medium for the communication. I am calling it the MessageQueueAppender (MQA). MQA is used by the 20 processes, which can easily log in log4cxx and the MQA will serialize the LogEvents into the message_queue. But know, what and how to implement a log4cxx-class which will read this message_queue LogEvents and pass it to its own logger? I will call it the MQ-Sink. (reading and deserializing from the message_queue is not the problem this is also already solved)

If I do the stupid way, I will loose the timestamp, the NDC and also the format of the serializedLogEventFromMessageQueue.

LogEvent le = deserializeFromMessageQueue(serializedLogEventFromMessageQueue);
LoggerPtr logger ...

So could anyone give me hints how to solve my problems?

As I said, I am willing to implement an MQA and also the MQ-Sink? But the MQ-Sink seems to be my biggest pain. And of course I am also willing to make the code again open source.

And the last question, but not mission cirtical. Is it possible to interoperate log4cxx with log4j via the message_queue?