---
blogpost: true
language: en
date: 2023-03-07
tags: til
author: felix
---
# TIL: Loki LogQL Pattern Matching for uWSGI Logs
Suppose you have log lines like this as uWSGI generates them:
```
[pid: 13|app: 0|req: 8331/12326] 172.20.0.11 () {34 vars in 999 bytes} [Tue Mar  7 16:08:57 2023] GET /foo => generated 4289 bytes in 45 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 1)
```

First, let's show the logs in a "Logs" view and filter them:
```
{instance="prod",compose_service="core"}
|= "generated"
!= "/metrics"
```

Extract fields by using the [pattern parser][] [^pattern-blog]:
```
{instance="prod",compose_service="core"}
|= "generated"
!= "/metrics"
# [pid: 13|app: 0|req: 8331/12326] 172.20.0.11 () {34 vars in 999 bytes} [Tue Mar  7 16:08:57 2023] GET /foo => generated 4289 bytes in 45 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 1)
| pattern `[<_>] <_> () {<_>} [<_>] <method> <path> => <_> (HTTP/1.1 <status>)`
```

You can now expand the line inside the explore view.
There you will see the extracted log labels `method`, `path` and `status`.

[pattern parser]: https://grafana.com/docs/loki/next/logql/log_queries/#pattern
[^pattern-blog]: https://grafana.com/blog/2021/08/09/new-in-loki-2.3-logql-pattern-parser-makes-it-easier-to-extract-data-from-unstructured-logs/


Finally, format the lines using the extracted fields:
```
{instance="prod",compose_service="core"}
|= "generated"
!= "/metrics"
# [pid: 13|app: 0|req: 8331/12326] 172.20.0.11 () {34 vars in 999 bytes} [Tue Mar  7 16:08:57 2023] GET /foo => generated 4289 bytes in 45 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 1)
| pattern `[<_>] <_> () {<_>} [<_>] <method> <path> => <_> (HTTP/1.1 <status>)`
| line_format `{{ .method }} {{ .path }} {{ .status }}`
```

This results in nicely readable output:
```
GET /foo 200
```


## Bonus
To filter by HTTP 4xx and 5xx error codes, you can use the following hack:
```
{instance="prod",compose_service="core"}
|= "generated"
!= "/metrics"
# [pid: 13|app: 0|req: 8331/12326] 172.20.0.11 () {34 vars in 999 bytes} [Tue Mar  7 16:08:57 2023] GET /foo => generated 4289 bytes in 45 msecs (HTTP/1.1 200) 2 headers in 73 bytes (1 switches on core 1)
| pattern `[<_>] <_> () {<_>} [<_>] <method> <path> => <_> (HTTP/1.1 <status>)`
| line_format `{{ .method }} {{ .path }} {{ .status }}`
# poor man's "status >= 400" follows, because there are no conditionals in golang's text/template 
# levels with color code: https://grafana.com/docs/grafana/latest/explore/logs-integration/#log-level
# if / else with hasPrefix: https://stackoverflow.com/a/69976898/241240
| label_format level=`{{ if hasPrefix "4" .status }}error{{else if hasPrefix "5" .status}}error{{else}}info{{end}}`
| level = "error"
```