FAQ

This document covers basic questions about Loggly's logging service. If you have a question that doesn't appear here, please visit the forums or make the addition yourself by creating an account on the wiki.

As one of the collection methods, Loggly runs a syslog based logging service. In the most simple terms, if you point your syslog server at logs.loggly.com, we'll catch and eat those logs and put them in your account for you. There are instruction below for configuring syslog, syslog-ng, rsyslog, Splunk, and other systems to send us logs. Check the logging configurations below for more info on how to get your logs to us.

Loggly can operate Syslog in two modes. The normal mode is where we index exactly what we receive via syslog. The other mode, the “stripping” mode, is where we eliminate the syslog headers before we index and store the data. Let me explain with some examples: Assume you are sending your data via straight syslog. You configured your device to log via syslog to Loggly. In this case, you would like to preserve all the syslog headers that the original syslog added to the event. An example of an event that you log and is going to be indexed in Loggly looks as follows:

Feb 16 15:33:22 app-1 collectd[1606]: uc_insert: Value too old: name =  i-d55fbbbe/df/df-dev-shm; value time = 1266363202; last cache update =  1266363202;  

This is the normal case and you would use the input type of “Syslog UDP” or “Syslog TCP”. If you configured “Syslog UDP Header Stripping” or the TCP analogous entry, you would end up with the following event being indexed in Loggly (note how the header got stripped off):

uc_insert: Value too old: name =  i-d55fbbbe/df/df-dev-shm; value time = 1266363202; last cache update =  1266363202;  

The case where you should be using the “Syslog UDP Header Stripping” is where you are tunneling events through syslog. For example, if you are reading from a file with SyslogNG and forward the content of the file to Loggly, you want to enable the stripping option. This is the case, for example when you monitor Apache logs. The following show what happens when you don't enable header stripping:

Feb 16 15:33:43 app-1 apache: 10.214.46.230 - - [16/Feb/2010:15:33:42 -0800] "GET /wp-content/themes/optimize/includes/js/reflectionXXX.js HTTP/1.1" 404 2326 "http://www.loggly.org/" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-us) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10" KgR2G66BhkYAAEcPAwYAAAAD 228199  

Notice how you end up with an additional syslog header. You don't want that. That is why you use the “Syslog UDP Header Stripping option to preserve only the original part of your log entry:

10.214.46.230 - - [16/Feb/2010:15:33:42 -0800] "GET /wp-content/themes/optimize/includes/js/reflectionXXX.js HTTP/1.1" 404 2326 "http://www.loggly.org/" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-us) AppleWebKit/531.21.8 (KHTML, like Gecko) Version/4.0.4 Safari/531.21.10" KgR2G66BhkYAAEcPAwYAAAAD 228199  

The short answer is that you should use TCP. The difference between UDP and TCP is that UDP is not reliable. Each packet is sent from your computer to our proxy server. If UDP is used, it could happen that the packet gets lost and therefore your log entry. In the case of TCP, it is guaranteed that the packet makes it to the proxy. If it should get dropped on the way, TCP will resend the packet and make sure that your log record arrives at Loggly. In terms of security, you are safer using TCP as well. It is not easy to spoof TCP messages, whereas UDP messages are incredibly simple to spoof.

You can find instructions on how to do this in the section on Logging Configurations.

You can think of input names as a way of more finely slicing up the data within all the data you send us. You should use names that clearly identify what kind of data is being handled by the input. We do some processing of the names as we index them, which allows you to search for data across multiple related inputs very easily. To get the full details on this, take a look at the Input Names page.

When you create an input, it is automatically in discovery mode, which means that you can send logs to that port from any machine you like. After 15 minutes, the input will fall out of discovery mode. It basically blocks any connection to that port. Except for the machines that have been sending data to that port during the period that the input was in discovery mode. This helps prevent that other people can send data to your inputs.

At any point in time, you can click on the “discover” slider to put an input back into discovery mode for 15 minutes. The machines that are allowed to send data to an input are called “devices”. You can see a list of devices that are allowed to send data to an input on the input detail page.

No. Please see the timestamp page which discusses how Loggly uses timestamps.

Make sure on a delete request to send the parameters as GET parameters, not POSTs:

curl -X DELETE -u <user>:<pass> http://<subdomain>.loggly.com/api/devices/?ip=10.0.0.3&input_id=879

or

curl -X DELETE -u <user>:<pass> http://<subdomain>.loggly.com/api/devices/10.0.0.3/

Make sure to check the API Documentation for how to access and manipulate the objects in the system.