FeatherCast

The voice of The Apache Software Foundation

Samza Portable Runner for Beam Hai Lu

September 13, 2019
timothyarthur

In this talk we are going to cover how we have leveraged portability of Beam and make Stream Processing in Python possible on top of Apache Samza. We will first touch points on Apache Samza in general, and how it stands out as the stream processing engine at LinkedIn that scales to over a trillion messages processed per day, with strong state support and flexible deployment models. Next we introduce Samza Runner for Beam, particularly the portable runner. We will cover the efforts we have done to leverage Beam and make stream processing in Python available at LinkedIn, followed by a few use cases. The talk will conclude with our plan for future work.

Leave a Reply

Required fields are marked *.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.
%d bloggers like this: