Application log aggregation, management and notifi

2020-05-19 03:02发布

站内文章 / 前沿技术

60 0

叛逆

女 | 书童

私信

可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效，请关闭广告屏蔽插件后再试):

问题:

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.

Want to improve this question? Update the question so it's on-topic for Stack Overflow.

Closed 4 years ago.

I'm wondering what everyone is using for logging, log management and log aggregation on their systems.

I am working in a company which uses .NET for all it's applications and all systems are Windows based. Currently each application looks after its own logging and notifications of failures (e.g. if app A fails it will send out its own 'call for help' to an admin).

While this current practice works its a bit hacky and hard to manage. I've been trying to find some options for making this work better and I've come up with the following:

log4net & Chainsaw (ah, if it works).
Logging via log4net or another framework into a central database & rolling our own management tool.
Logging to the Windows event log and using MOM or System Center Operations Manager to aggregate and manage each of these servers & their apps.
A hand-rolled solution to suck all the log files into one point and work some magic across them.

Essentially what we are after is something which can pull log entries all together and allow for some analytics to be run across them, plus use a kind of event based system to, for example, send out a warning email when there have been 30+ warning level logs for an application in the last x minutes.

So is there anything I've missed, or something someone else can suggest?

回答1:

If you can, I'd recommend writing to the EventLog and creating rules in SCOM to monitor. We use this extensively and it works well, even to a point of putting together pieces of code which monitor certain elements of our apps and writing values to the event log, where SCOM parses for the errors, and graphs those, plus informational errors, into reports showing stats over a given time.

I am however quite keen on rewriting some that into WMI, and having SCOM poll the WMI service for those same counters, as writing queue lengths to event log every 15 minutes seems a little wasteful ;)

回答2:

L4ndash: log4net log viewer with free developer version.

回答3:

We've used MOM with pretty good success since it handles the purging, viewing, and other concerns outside of data entry. The caveat to MOM is that it doesn't do *NIX out of the box(at least the version we have). There are connectors and such. Another option is look into LogLogic. This takes log info from network devices, but has an open web service API that you could tie into. MOM has APIs as well, so you could funnel down to something like LogLogic.

回答4:

For log aggregation, Snare is a little old but worth mentioning.

Snare Agent for IIS Servers

I used this old version a long time ago and it worked well by forwarding/sending/replicating IIS logs over a network via syslog.

Today, they have a newer version called Snare Epilog

The code is also open source; you might want to give it a try.

Also, you could use something like ELMAH combined with the RSS/Email features to help notify and alert you of any critical issues.

回答5:

We are using ErrorsDigest Simple real time errors aggregation.

回答6:

I think we've got something very close to what you're looking for. logFaces is a log server, aggregator and viewer. It's built particularly for larger distributed applications and multi-user environments. We put together all the good stuff from Apache logging services and made out-of-the-box solution for those who use log4j, log4net, log4cxx or event log4python. logFaces can work with major database brands and will let you quickly dig out the issues from the flood of log data which is very common in large applications. It will also let you see the slices of your system (log-wise) in real-time. There are many more features, feel free to give it a try. If you like I can help you with setup directly, although it's very simple to get started just after watching the demos we have.

Disclosure: I am the author of this product.

回答7:

Check the Microsoft Log Parser and examples. The log parser can handle xml, csv as well as Microsoft event log, registry and can send information to 'syslog' servers.

From the example screenshot this seems to be really useful tool.

回答8:

It depends on your needs. If, for example, you're running a data center and are mostly concerned with real-time alarming, then you might want to look at Avicode. It's very capable, but can be pricey.

If, on the other hand, you're supporting desktop applications or logs from remote systems, or are more concerned with offline log analysis then you might take a look at Gibraltar. It provides a nice combination of logging, log management and log aggregation/analysis capabilities. But, to be honest, I'm a bit partial -- I'm one of the Gibraltar developers.

回答9:

log4net is definitely the place to go for simple logging that outputs to the most common targets. log4net can log to windows eventlog as well as put messages out to db, email or file. It's fairly trivial to setup and the documentation is pretty good.

回答10:

I'm using log4net and log4view (www.log4view.org). They have a TCP adapter that you can distribute as part of your application. You then use their client app to connect to your various applications and the log messages are displayed in real time.

It can monitor multiple logs at the same time - up to 255 if you but the commercial version (a measly $50).

It can also work with the RollingFileAppender.

They have a pretty powerful filtering UI so you can home in on what you want to look at.

回答11:

First, use an established and widely used logging framework like log4net or NLog.

Both of the logging frameworks can send log messages to multiple destinations, e.g. to a log file, a network stream (TCP or UDP) or a database.

Then use a log viewer like Log4View to visualize and filter the log messages. If you use Log4View, please make sure that you create XML formatted log files. Although Log4View can also parse (human readable) pattern formatted log files, XML formatted log data is easier to configure and faster to parse.