Selfhosted

40135 readers

1412 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago

MODERATORS

HybridSarcasm@lemmy.world

HybridSarcasm@lemmy.hybridsarcasm.xyz

Parsing RFC3164 logs for Grafana (lemmy.sdf.org)

submitted 1 year ago by wgs@lemmy.sdf.org to c/selfhosted@lemmy.world

9 comments fedilink hide all child comments

Hey everyone ! I finally decided to monitor my applications more closely with Grafana. However I'm having issues building dashboards their logs.

Their logs are currently sent over syslog (in RFC3164 format) into telegraf. But it simply puts the whole message into the message field, so I can't use specific fields (eg. URL for httpd, source IP for DNS requests, username for SSH, …) to build graphs.

I've read about grok patterns, but I have no idea how to use them.

Would someone have any pointer on how I could make sense out of these logs for later use ?

you are viewing a single comment's thread
view the rest of the comments

[–] vegetaaaaaaa@lemmy.world 4 points 1 year ago* (last edited 1 year ago) (1 children)

I have a similar setup (all hosts sending logs through syslog protocol to a central collector), but the collector is graylog. A few years back it used to use Grok expressions, but now it has its own filter syntax. My notes on extractors/grok patterns are still there (unfold details). Can't help you much more than that, sorry!

[–] wgs@lemmy.sdf.org 1 points 1 year ago

It does help thank you ;)

I've found that you can use custom grok patterns to parse logs just as grayling extractors do. I'm still trying to figure it out, but so far I could start parsing logs using a [[processor.parser]] block. I'll document my findings when I get it working as I want it.