Understanding CloudEvents: Simplifying Event Data Interoperability

5 min readMay 11, 2024

CloudEvents is a specification that aims to provide a common format for describing event data. It allows services to produce or consume events independently, increasing portability and interoperability. It dramatically simplify event declaration and delivery across services, platforms, and beyond.

CloudEvents was mentioned in thoughtworks techradar Apr 2024 edition under Adopt category from Trial category in Sep 2023 and from Assess Category in Apr 2019

Brief History

The CNCF Serverless Working group was originally created by the CNCF’s Technical Oversight Committee to investigate Serverless Technology and to recommend some possible next steps for some CNCF related activities in this space. One of the recommendations was to investigate the creation of a common event format to aid in the portability of functions between Cloud providers and the interoperability of processing of event streams. As a result, the CloudEvents specification was created.

Concepts of Cloud Events

An event contains information and details about something that happened. Each happening is distinguished by the details in the event.

Events show what happened and do not have a specific information about where to deliver it, while messages carry a purpose, moving information from a sender to a specific receiver intended for delivery.

Eventing

Events are very common in modern software systems with Event Driven architecture. For example, a source may generate an event when it receives an external signal (e.g. HTTP or RPC) or observes a changing value(e.g. an IoT sensor sending sample of temperature value).

Events are everywhere, yet event publishers tend to describe events differently.

Let us consider a use case where an airline company notifies about newly created flight schedules, change in flight timings to various service providers and/or online travel booking websites for users to get updated information. Think of the scenario, each of the system and online travel booking sites consume the infomation are hosted in various cloud providers and ingest it via Kafka , Event grids, MQTT or through Cloud Providers Function API. Image below depicts it

For a sample message below,

{
  "flightNo":"AE987",
  "airlines":"X Airlines",
  "scheduledDeparture":"1450"
}

the airline system has to format the message for Azure Event grid, Kafka and a HTTP Post message for various downstream systems to consume.

Azure Event Grid Messsage

{
  "topic": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.EventGrid/topics/{event-grid-topic-name}",
  "subject": "FlightUpdate",
  "eventType": "FlightScheduleUpdated",
  "id": "{unique-event-id}",
  "data": {
    "flightNo": "AE987",
    "airlines": "X Airlines",
    "scheduledDeparture": "1450"
  },
  "eventTime": "2022-01-01T12:00:00.000Z",
  "dataVersion": "1.0"
}

AWS Managed Kafka

{
  "Records": [
    {
      "Value": "{\"flightNo\":\"AE987\",\"airlines\":\"X Airlines\",\"scheduledDeparture\":\"1450\"}",
      "PartitionKey": "partitionKey-1"
    }
  ]
}

HTTP Post Message

POST /your-endpoint HTTP/1.1
Host: your-hostname
Content-Type: application/json
Content-Length: [length of the JSON data]

{
  "flightNo": "AE987",
  "airlines": "X Airlines",
  "scheduledDeparture": "1450"
}

The Airline system has to format and construct message for various downstream systems in different formats for it to ingest.

How it would be if we have a d message format to send the same data across these downstream systems?

You guessed it right… Cloudevents play that role.

CloudEvents is a specification for describing event data message in a common way. CloudEvents seeks to dramatically simplify event declaration and delivery across services, platforms, and beyond!

It simplifies the way airline systems publishes the message to the downstream systems like below, all messages in Cloud Event specification.

Sample CloudEvents message

{
  "specversion":"0.3",
  "type":"com.airline.schedulechange",
  "source":"https://airline.com/flights/schedule/AE780",
  "id":"70d3c768-63f8-40e7-aa9d-d197d530586b",
  "time":"2019-07-04T17:31:00Z",
  "datacontenttype":"application/json",
  "data":{
    "flightNo": "AE987",
    "airlines": "X Airlines",
    "scheduledDeparture": "1450"
  }
}

Lets write some code

The CloudEvents specification offers guidance on how to handle evolving data schemas within events while maintaining version compatibility.

The ‘type’ attribute plays a key role in helping consumers identify the type of event they receive. It should remain consistent for backwardly-compatible changes in event data and change when modifications are backwardly-incompatible. This would help way on how to parse the data in Cloud Event.

The ‘dataschema’ attribute serves an informational purpose, aiding in development and tooling support.

The Cloudevents ‘id’ core attribute in serves as a unique identifier for events from the same source, ensuring no duplication. Its primary function is to distinguish between different occurrences and correlate related events within the context of a single event source.

Cloudevents extension attributes to the CloudEvent specification are meant to be additional metadata that needs to be included to help ensure proper routing and processing of the CloudEvent.

By understanding and implementing these considerations, event producers can effectively manage schema evolution and versioning in CloudEvents, ensuring smooth communication and compatibility with consumers.

Cloudevents provide SDKs for

In this blog we will see some example using JavaScript, and expressjs. First things first let us initialize a new project using npm init and install minimal dependencies for this example

npm install express body-parser cloudevents

Now let us create CloudEvent consumer

Cloud Event Emitter Code

Curl command to publish and consumer message in the above code

curl --location 'http://localhost:4000/' \
--header 'Content-Type: application/cloudevents+json' \
--data '{
  "specversion":"0.3",
  "type":"com.airline.schedulechange",
  "source":"https://airline.com/flights/schedule/AE780",
  "id":"70d3c768-63f8-40e7-aa9d-d197d530586b",
  "time":"2019-07-04T17:31:00Z",
  "datacontenttype":"application/json",
  "data":{
    "flightNo": "AE987",
    "airlines": "X Airlines",
    "scheduledDeparture": "1450"
  }
}'

Postman Output

Below are some of the advantages for CloudEvents specification, hosted by the Cloud Native Computing Foundation (CNCF)

Interoperability: Enables seamless communication between different event-driven systems and platforms.
Portability: Facilitates easy integration and data exchange across diverse environments.
Standardization: Provides a common format for describing event data, promoting consistency and compatibility in event-driven architectures.
Evolution: Supports schema evolution and versioning, allowing for flexible and sustainable development.
Collaboration: Encourages a shared space for documenting extensions, fostering collaboration and innovation in the event-driven ecosystem.

Below are the supported CloudEvents adapter

Finalthoughts

CloudEvents offers a standardized approach to handling event data across different platforms and services, simplifying integration, improving portability, and enhancing development and testing in event-driven architectures. By supporting versioning, promoting consistency in event formats, and facilitating event correlation, CloudEvents addresses key challenges in event processing, ensuring seamless communication and efficient workflow management in diverse environments.

Understanding CloudEvents: Simplifying Event Data Interoperability

Brief History

Concepts of Cloud Events

Eventing

References

Written by Meiyappan Kannappa

No responses yet