Ingestion best practices

The Logs schema in Obics is loose. You can do whatever fits your company, which often means adhering to existing conventions. We do recommend a few best practices, which you can choose to implement.

Recommended fields/columns

  1. The level field is important because Obics will use it to automatically identify problems and create alerts. (alerts functionality coming soon). That's why we recommend using it over any other custom column you might already have like severity.
  2. Consider using OpenTelemetry standard and include fields like traceId and spanId (or trace_id and span_id). Even if you're not starting with OpenTelemetry, you might choose to use it in the future.
  3. Add any correlation IDs like web session IDs. Obics allows you to join logs, metrics, and traces and if you have a dedicated correlation field to join on, you can to make the most use of it.
  4. Since Obics can join log data, you don't have to include metadata in every single log. Instead, you can send request-relevant metadata in the first log of a request, and trace-relevant metadata in the first log of a trace.
  5. Consider making use of static log identifiers in the form of a event_id field. In an OLAP system like Obics, they are very helpful for aggregations and join operations.

Example Schemas

For a web application that consists of a browser client side and a single backend API, a logging schema might be something like this:

Field NameTypeDescription
timeDateTimeThe timestamp indicating when the log entry was created, sent as a unix timestamp.
levelUInt8Log level (1=TRACE, 2=DEBUG, 3=INFO (default), 4=WARN, 5=ERROR, 6=FATAL).
messagestringMain log message or description of the event. Plain text or JSON.
event_idstringA unique identifier for the log. Written statically in the log statement in code.
servicestringName of the service or application generating the log. In this case, the browser or the API backend.
hoststringHostname or IP address of the machine generating the log.
traceIdstringA unique identifier for a request/transaction that stays the same across multiple services.
spanIdstringA unique identifier for each span within a trace.
browserSessionIdstringThe browser web session. Will be empty for non-browser logs.
envrionmentstringEnvironment where the log originated (production, staging, development).

Here's an example of an entry in JSON, in same format as it should be sent with Obics REST API:

{
  "time": 1735282088,
  "message": "Failed to retrieve user data from database",
  "level": 5,
  "event_id": "usrD1",
  "service": "user-service",
  "host": "web-server-1",
  "traceId": "123456789abcdef",
  "spanId": "abcdef123456789",
  "browserSessionId": "bcd433b8-f4c9-426b-bf61-6558cd0dcabd",
  "environment": "production",
}
Ingestion best practices | Obics