Logging

Log Analysis

PaperTrailApp.com
LogEntries.com
Splunk
GrayLog
Elk
https://logit.io/
https://play.grafana.org/d/000000012/grafana-play-home?orgId=1
https://www.elastic.co/elk-stack
https://www.elastic.co/webinars/introduction-elk-stack
https://logz.io/learn/complete-guide-elk-stack/
https://blog.overops.com/splunk-vs-elk-the-log-management-tools-decision-making-guide/
https://www.digitalocean.com/community/tutorial_series/centralized-logging-with-elk-stack-elasticsearch-logstash-and-kibana-on-ubuntu-14-04
https://medium.com/@brunoamaroalmeida/enabling-centralized-application-logging-using-the-elastic-elk-stack-from-stratch-a-15-minutes-eba501230b3d

https://logback.qos.ch/manual/introduction.html
https://help.papertrailapp.com/kb/configuration/java-logback-logging/#syslog4jappender-recommended
https://logback.qos.ch/manual/configuration.html
https://github.com/papertrail/logback-syslog4j
https://examples.javacodegeeks.com/enterprise-java/logback/logback-configuration-example/
https://stackify.com/logging-logback/
https://www.javacodegeeks.com/2012/04/using-slf4j-with-logback-tutorial.html
https://memorynotfound.com/logback-logback-xml-configuration-example/
https://dzone.com/articles/logging-java-switching-logback

https://www.bugsnag.com/platforms/javascript/?
Airbreak

https://www.sitepoint.com/how-can-the-elk-stack-be-used-to-monitor-php-apps/ - open-sourced

https://www.graylog.org/
http://graylog2.org/

https://blog.logentries.com/2016/05/logging-for-continuous-integration/

https://www.sitepoint.com/premium/screencasts/handling-php-logging-with-monolog

http://logexpert.codeplex.com/
http://sourceforge.net/projects/gamutlogviewer/
http://ophilipp.free.fr/op_tail.htm
http://insightextensions.codeplex.com/
http://www.baremetalsoft.com/baretail/
http://glogg.bonnefon.org/description.html
http://logview4net.com/
http://sourceforge.net/projects/logview4net
http://www.moonlit-software.com/
http://baudlabs.com/top-free-and-open-source-log-management-software/
http://alternativeto.net/software/logviewer/
https://logging.apache.org/chainsaw/
http://www.findbestopensource.com/tagged/log-analysis
http://www.log4view.com/log4view/
http://www.moonlit-software.com/
http://jlogviewer.sourceforge.net/
http://sourceforge.net/projects/jlogviewer/
http://www.xpolog.com/
http://sourceforge.net/projects/mindtreeinsight
https://plus.google.com/u/0/102275357970232913798/posts/Fsu6qftH2ja
http://sourceforge.net/projects/lilith/
http://lilithapp.com/
https://www.npmjs.com/package/log-color-highlight
https://www.npmjs.com/package/file-tail

http://www.sitepoint.com/logging-with-monolog-from-devtools-to-slack/
https://www.loggly.com/
http://www.sitepoint.com/effective-php-logging-loggly

http://logio.org/

http://www.splunk.com/
http://www.splunk.com/view/SP-CAAADFV#difference
http://www.splunk.com/view/SP-CAAAE8W
http://www.splunk.com/download
http://docs.splunk.com/Documentation/Splunk/6.0.1/Admin/MoreaboutSplunkFree
http://www.splunk.com/view/pricing/SP-CAAADFV
http://docs.splunk.com/Documentation/Splunk/6.0.1/Admin/TypesofSplunklicenses

Apache Chainsaw log viewer, which can be downloaded from http://logging.apache.org/chainsaw/index.html.

http://docs.fluentd.org/articles/free-alternative-to-splunk-by-fluentd
http://www.quora.com/What-are-the-best-free-alternatives-to-Splunk
Kiwi Log Viewer
https://www.logrhythm.com/
https://www.sumologic.com
http://www.manageengine.com/products/eventlog/eventlog-analysis.html
http://www.sawmill.net
https://logentries.com/
http://nxlog.org/why-nxlog
http://www.balabit.com/network-security/syslog-ng/opensource-logging-system
http://www.intersectalliance.com/projects/
http://www.logalyze.com/

http://javadiff.sourceforge.net/jdiff/reports/j2se140_j2se141_docs/changes/java.util.logging.Level.html
http://en.wikipedia.org/wiki/Java_logging_framework
http://www.onjava.com/pub/a/onjava/2002/06/19/log.html?page=2
https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Level.html
https://docs.oracle.com/javase/7/docs/api/java/util/logging/Level.html
http://tutorials.jenkov.com/java-logging/levels.html
http://www.javapractices.com/topic/TopicAction.do?Id=143
http://stackoverflow.com/questions/5817738/how-to-use-log-levels-in-java
http://www.vogella.com/tutorials/Logging/article.html

http://netlogger.lbl.gov/
https://docs.google.com/document/d/1oeW_l_YgQbR-C_7R2cKl6eYBT5N4WSMbvz0AT6hYDvA/edit
http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/UnderstandandusetheCommonInformationModel
https://www.manageengine.com/products/eventlog/ELA_Best_Practices_Guide.pdf
http://www.sqlmag.com/article/event-logs/ssis-logging-best-practices
http://c2.com/cgi/wiki?LoggingBestPractices
http://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/LoggingErrorsAndWarnings.html
http://dev.splunk.com/view/SP-CAAADP6
http://www.alertlogic.com/wp-content/uploads/2012/01/Log-Management-Best-Practices.pdf
http://www.govhealthit.com/sites/govhealthit.com/files/resource-media/pdf/elm_-_compliance_best_practices_govt_-_healthcare.pdf
http://acs.lbl.gov/netlogger/snapshots/2011-02-24/manual.html
http://www.escholarship.org/uc/item/1jz4k8hd
http://www.nersc.gov/assets/Uploads/cedps-ts-NERSC.pdf
http://www.osti.gov/bridge/servlets/purl/932522-PF97Ml/932522.pdf
http://www.ibm.com/developerworks/java/library/j-logging/
http://blog.dynom.nl/archives/Logging-best-practices_20120304_63.html
http://nirajrules.wordpress.com/2008/06/14/blogging-best-practicies/
http://juliusdavies.ca/logging.html
http://www.splunklabs.com/logging/bestpractices.html
http://javarevisited.blogspot.com/2011/05/top-10-tips-on-logging-in-java.html
http://www.gibraltarsoftware.com/Support/Documentation/logging_bestpractices.html
http://forum.springsource.org/showthread.php?11845-Logging-best-practices
http://www.channeldb2.com/profiles/blogs/log-sizing-and-best-practices-for-logging-and-log-monitoring

Logging should use consistent structured format. Logging should include timestamp, and unique IDs (such as session ID, user ID, application ID, facility ID, etc). We may need to build our own logger that extend the base logging library. Structure and consistent logging makes program easier to debug and maintain.

Log level should be configured at reasonable level. There should be a way to dynamically change the log level at the container level, or individually change the log level at the user level. Verbose logging can severely impact application performance

What should be logged?

  • In general, anything that may take a long time, or fail should be logged.
  • Wherever possible, both the beginning and the end of an operation should be logged
  • Service initiation, configuration, and termination: Whenever a service starts up or a service request thread is launched, this should be logged. If the service can be configured, this message must contains a reference to the service configuration used. The termination message should include a status or termination message or code
  • Errors: All errors that cause a component to exit should be logged
  • Authentication and authorization operations: Authentication events should include the authentication method and claimed identity. Authorization events should include the remote identity, mechanism, and mechanism-specific attributes. We should also include a reason for authentication error. It may make sense to define different reason codes for different authentication methods – for example, standard X.509 authentication may have reasons like “certificate expired” or “certificate issued by untrusted CA”, while username/password authentication may have reasons like “unknown user” or “bad password”.
  • Remote Connection: When a log message pertains to an attempted connection to/from a remote service, the log should contain the IP address and port number.
  • Entering or exiting functions and major loops

How should events be logged?

The essential elements for constructing high-quality logs are as follows:

  • Structure and format: Consistently structure, self-describing, ASCI text records
  • Level of detail: Use of logging levels to seperate logs by details
  • Event types: Unique names for each logged event from a heirarchical namespace
  • Timestamps: High-resolution timestamps in a standard format
  • Identifier: Explicit and clearly labeled identifier for resources

When should we log?

  • At the beginning of each function. We should log the function name, and the values of simple parameters (except possibly for parameters that are objects, in which case we can test to see if the parameters are defined)
  • Every decision points
  • At the end of each function. We should log the name of the function, and its return value.

When should we not log?

  • Avoid logging in a tight loop

Why do we need to use logging?

Inserting log statements into code is a low-tech method for debugging it. It may also be the only way because debuggers are not always available or applicable. This is usually the case for multithreaded applications and distributed applications at large. Experience indicates that logging was an important component of the development cycle. It offers several advantages. It provides precise context about a run of the application. Once inserted into the code, the generation of logging output requires no human intervention. Moreover, log output can be saved in persistent medium to be studied at a later time. In addition to its use in the development cycle, a sufficiently rich logging package can also be viewed as an auditing tool.

What is the main drawback of logging?

It can slow down an application, especially when used inside a loop.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License