After restarting my "afserver", filebeat did not autostart.....seems I forgot:
sudo update-rc.d filebeat defaults 95 10
When I discovered this after semeral hours, I started it. But a log file was not colleted correct afterwards - in fact, filebeat does not log data from it:
Other logs seems ok:
But looking at the registry file in, I can see the status for the file:
"/var/log/ping-kaf.txt":{"source":"/var/log/ping-kaf.txt","offset":143350,"FileStateOS":{"inode":393591,"device":64513}}
And the offset (bytes read from the file) is growing:
18:36 142692 18:37 142786 18:42 143350 ... ... 18:57 144196
Filebeat logging (info) - no clue...
2016-04-04T18:52:24+02:00 INFO Harvester started for file: /var/log/ping-kaf.txt 2016-04-04T18:52:24+02:00 INFO Harvester started for file: /var/log/auth.log 2016-04-04T18:52:24+02:00 INFO Harvester started for file: /var/log/apache2/alfresco.mos-eisley.dk-access.log 2016-04-04T18:52:24+02:00 INFO Harvester started for file: /var/log/apache2/elk.mos-eisley.dk-error.log 2016-04-04T18:52:24+02:00 INFO File was truncated. Begin reading file from offset 0: /var/log/apache2/elk.mos-eisley.dk-error.log 2016-04-04T18:52:24+02:00 INFO Harvester started for file: /var/log/syslog 2016-04-04T18:52:24+02:00 INFO Harvester started for file: /var/log/apache2/elk.mos-eisley.dk-access.log 2016-04-04T18:52:24+02:00 INFO File was truncated. Begin reading file from offset 0: /var/log/apache2/elk.mos-eisley.dk-access.log 2016-04-04T18:52:24+02:00 INFO All prospectors initialised with 7 states to persist 2016-04-04T18:52:24+02:00 INFO Starting Registrar 2016-04-04T18:52:24+02:00 INFO Start sending events to output 2016-04-04T18:52:27+02:00 INFO Events sent: 2048 2016-04-04T18:52:27+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:52:32+02:00 INFO Events sent: 678 2016-04-04T18:52:32+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:52:44+02:00 INFO Events sent: 12 2016-04-04T18:52:44+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:53:07+02:00 INFO Events sent: 3 2016-04-04T18:53:07+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:53:14+02:00 INFO Events sent: 1 2016-04-04T18:53:14+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:54:07+02:00 INFO Events sent: 1 2016-04-04T18:54:07+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:54:12+02:00 INFO Events sent: 12 2016-04-04T18:54:12+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:54:27+02:00 INFO Events sent: 1 2016-04-04T18:54:27+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:55:04+02:00 INFO Events sent: 2 2016-04-04T18:55:04+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:55:12+02:00 INFO Events sent: 2 2016-04-04T18:55:12+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:55:49+02:00 INFO Events sent: 2 2016-04-04T18:55:49+02:00 INFO Registry file updated. 7 states written. 2016-04-04T18:56:04+02:00 INFO Events sent: 4 2016-04-04T18:56:04+02:00 INFO Registry file updated. 7 states written.
In debug mode:
2016-04-04T19:02:18+02:00 DBG File Configs: [/var/log/ping-kaf.txt] 2016-04-04T19:02:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:02:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:18+02:00 DBG Start harvesting unknown file: /var/log/ping-kaf.txt 2016-04-04T19:02:18+02:00 DBG Resuming harvester on a previously harvested file: /var/log/ping-kaf.txt 2016-04-04T19:02:18+02:00 DBG harvest: "/var/log/ping-kaf.txt" position:144572 (offset snapshot:0) 2016-04-04T19:02:18+02:00 INFO Harvester started for file: /var/log/ping-kaf.txt 2016-04-04T19:02:18+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:02:18+02:00 DBG Registrar will re-save state for /var/log/ping-kaf.txt 2016-04-04T19:02:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:02:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:18+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:18+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:02:19+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:02:21+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:02:25+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:02:28+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:02:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:02:33+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:02:38+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:02:38+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:38+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:38+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:02:43+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:02:48+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:02:48+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:48+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:48+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:02:53+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:02:58+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:02:58+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:58+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:02:58+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:03:03+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:03:04+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:03:06+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:03:08+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:03:08+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:08+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:08+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:03:10+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:03:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:03:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:18+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:18+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:03:18+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:03:28+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:03:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:03:28+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:03:38+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:03:38+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:38+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:38+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:03:38+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:03:48+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:03:48+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:48+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:48+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:03:48+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:03:58+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:03:58+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:58+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:03:58+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:03:58+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:04:08+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:04:08+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:04:08+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:08+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:08+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:04:09+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:04:11+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:04:15+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:04:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:04:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:18+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:18+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:04:23+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:04:28+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:04:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:04:33+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:04:38+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:04:38+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:38+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:38+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:04:43+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:04:48+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:04:48+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:48+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:48+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:04:53+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:04:58+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:04:58+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:58+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:04:58+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:05:03+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:05:04+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:05:06+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:05:08+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:05:08+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:08+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:08+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:05:10+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:05:18+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:05:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:05:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:18+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:18+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:05:28+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:05:28+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:05:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:05:38+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:05:38+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:05:38+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:38+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:38+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:05:48+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:05:48+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:05:48+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:48+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:48+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:05:58+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:05:58+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:05:58+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:58+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:05:58+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:06:08+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:06:08+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:06:08+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:08+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:08+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:06:09+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:06:11+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:06:15+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:06:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:06:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:18+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:18+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:06:23+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:06:28+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:06:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:06:33+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:06:38+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:06:38+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:38+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:38+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:06:43+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:06:48+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:06:48+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:48+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:48+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:06:53+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:06:58+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:06:58+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:58+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:06:58+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:07:03+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:07:04+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:07:06+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:07:08+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:07:08+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:08+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:08+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:07:10+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:07:18+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:07:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:07:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:18+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:18+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:07:28+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:07:28+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:07:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:07:38+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:07:38+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:07:38+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:38+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:38+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:07:48+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:07:48+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:07:48+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:48+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:48+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:07:58+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:07:58+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:07:58+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:58+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:07:58+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:08:08+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:08:08+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:08:08+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:08+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:08+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:08:09+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:08:11+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:08:15+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:08:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:08:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:18+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:18+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:08:23+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:08:28+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:08:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:08:33+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:08:38+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:08:38+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:38+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:38+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:08:43+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:08:48+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:08:48+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:48+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:48+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:08:53+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:08:58+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:08:58+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:58+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:08:58+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:09:03+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:09:04+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:09:06+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:09:08+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:09:08+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:08+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:08+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:09:10+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:09:18+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:09:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:09:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:18+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:18+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:09:28+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:09:28+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:09:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:09:38+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:09:38+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:09:38+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:38+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:38+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:09:48+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:09:48+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:09:48+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:48+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:48+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:09:58+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:09:58+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:09:58+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:58+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:09:58+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:10:08+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:10:08+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:10:08+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:10:08+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:10:08+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:10:09+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:10:11+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:10:15+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:10:18+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:10:18+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:10:18+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:10:18+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt 2016-04-04T19:10:23+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now. 2016-04-04T19:10:28+02:00 DBG scan path /var/log/ping-kaf.txt 2016-04-04T19:10:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:10:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt 2016-04-04T19:10:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt root@afserver:~#
The phrase
Not harvesting, file didn't change: /var/log/ping-kaf.txt
is simple not correct, the file changes every minute:
Mon Apr 4 19:02:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=12.7 ms Mon Apr 4 19:03:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=12.0 ms Mon Apr 4 19:04:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=14.4 ms Mon Apr 4 19:05:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=12.0 ms Mon Apr 4 19:06:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=11.9 ms Mon Apr 4 19:07:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=11.7 ms Mon Apr 4 19:08:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=12.9 ms Mon Apr 4 19:09:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=12.3 ms Mon Apr 4 19:10:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=12.5 ms Mon Apr 4 19:11:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=12.2 ms
After truncating the log file
cat /dev/null > /var/log/ping-kaf.txt
Filebeat found the truncation and started reading:
2016-04-04T19:14:23+02:00 DBG File was truncated as offset (%!s(int64=145700)) > size (%!s(int64=0)). Begin reading file from offset 0: /var/log/ping-kaf.txt
2016-04-04T19:14:23+02:00 INFO File was truncated. Begin reading file from offset 0: /var/log/ping-kaf.txt
2016-04-04T19:14:23+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now.
2016-04-04T19:14:28+02:00 DBG Flushing spooler because of timeout. Events flushed: 0
2016-04-04T19:14:28+02:00 DBG Start next scan
2016-04-04T19:14:28+02:00 DBG scan path /var/log/ping-kaf.txt
2016-04-04T19:14:28+02:00 DBG Check file for harvesting: /var/log/ping-kaf.txt
2016-04-04T19:14:28+02:00 DBG Update existing file for harvesting: /var/log/ping-kaf.txt
2016-04-04T19:14:28+02:00 DBG Not harvesting, file didn't change: /var/log/ping-kaf.txt
...
...
...
...
...
2016-04-04T19:18:15+02:00 DBG End of file reached: /var/log/ping-kaf.txt; Backoff now.
4-04T19:18:16+02:00 DBG Flushing spooler because of timeout. Events flushed: 3
2016-04-04T19:18:16+02:00 DBG Publish: {
"@timestamp": "2016-04-04T17:18:08.976Z",
"beat": {
"hostname": "afserver",
"name": "afserver"
},
"count": 1,
"fields": null,
"input_type": "log",
"message": "Mon Apr 4 19:18:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=11.4 ms",
"offset": 282,
"source": "/var/log/ping-kaf.txt",
"type": "pinglog"
}
...
...
...
...
...
016-04-04T19:21:08+02:00 DBG Publish: {
"@timestamp": "2016-04-04T17:21:03.983Z",
"beat": {
"hostname": "afserver",
"name": "afserver"
},
"count": 1,
"fields": null,
"input_type": "log",
"message": "Mon Apr 4 19:21:01 CEST 2016: 64 bytes from 194.88.5.66: icmp_seq=1 ttl=245 pingtime=12.2 ms",
"offset": 564,
"source": "/var/log/ping-kaf.txt",
"type": "pinglog"
}