I'm dealing with an application using multiple microservices.
The application receieves data and processes it, and I'd like to have a way to monitor everything the received data is going throughout the microservices.
Since the platform is nodejs we're dealing with asynchronous processing, logs wont always be written one after the other, and since the data does not have aמ ID the logs cannot be filtered in any convenient way.
Is there any way to track the data from the first moment it enters the application till the end, moving between the microservices without having to pass a generate id between all the methods.
Obviously that id will be passed between the microserives but within each one, I'd like to keep the code clean.
I'm using winston to log, maybe something could fit there.
Thanks in advance, any suggestion would be highly appreciated.
By framework, rsp meant Express, Hapy, Restify, etc. Generically speaking, the idea would be that the service receives the request id in a header (x-request-id). For each request a logger instance is initialised with the id which it will include in any logs.
In your app you need a way of accessing the single logger wherever you want to log from. I would suggest logging only happens in the route handlers so attaching the logger to the request object would likely be the way to go.
Restify handles a lot of this by default. Hapi would need a plugin, and for Express you would use middleware.
You will also then need to ensure that any requests to your other services include the request id header.
Is there any way to track the data from the first moment it enters the application till the end, moving between the microservices without having to pass a generate id between all the methods.
No, because none of your microservices could know which of the other requests are related to the current one, unless you generate some id and pass it between all of them.
If you add some request ID that is optional, then the microservise that don't have it in the request may generate it randomly (using a UUID for example) and then include it in all of the logs and pass it in requests to all other microservices, that in turn would use the same ID instead of generating a random one.
Without such an ID you will not have enough context to group all of the requests together, especially between different microservices. Depending on the framework you're using it can be very easy to write a middleware to handle that for you. Since you didn't say which framework you're using it's impossible to give you a more specific answer.
