Skip to main content
OpenObservability Talks

OpenObservability Talks

By Dotan Horovits

On OpenObservability Talks we discuss harnessing the power of open source to advance observability initiatives for DevOps practitioners around the world.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
www.youtube.com/@openobservabilitytalks

You can find us on Twitter @openobserv
Available on
Apple Podcasts Logo
Google Podcasts Logo
Overcast Logo
Pocket Casts Logo
RadioPublic Logo
Spotify Logo
Currently playing episode

Charting New Territory: OpenTelemetry Embraces Profiling - OpenObservability Talks S4E10

OpenObservability TalksMar 28, 2024

00:00
01:06:45
Charting New Territory: OpenTelemetry Embraces Profiling - OpenObservability Talks S4E10

Charting New Territory: OpenTelemetry Embraces Profiling - OpenObservability Talks S4E10

OpenTelemetry is expanding beyond the traditional “three pillars of observability” and introduces a groundbreaking addition to its signals - Continuous Profiling. The new Profiling Special Interest Group (SIG) that was formed to lead the topic has already made significant advancements, to be featured at KubeCon Europe. Join us in this special panel episode of OpenObservability Talks as we explore the significance of this new dimension in understanding application behavior, optimizing performance, and gaining deeper insights into your systems. Our expert guests, Felix Geisendörfer and Ryan Perry, members of the OpenTelemetry Profiling SIG, share their insights into how Profiling enhances the OpenTelemetry framework, and update on the work for open specification and implementation. 


This special episode hosts a panel of two distinguished members of OpenTelemetry’s Profile SIG, and prominent members of the observability vendor ecosystem. Felix Geisendörfer is a Senior Staff Engineer at Datadog where he works on Continuous Profiling and contributes to the Go runtime. Before that he was working at Apple, co-founded Transloadit, contributed to node.js and inspired a generation of mad scientists to program flying robots with it. Ryan Perry is Principal Product Manager at Grafana Labs. He has built a career at various startups while actively contributing to open source projects and advancing open telemetry initiatives. Most recently he built Pyroscope, an open source continuous profiling project/company, which has been acquired by Grafana Labs.

The episode was live-streamed on 7 March 2024 and the video is available at https://www.youtube.com/watch?v=iGM67RT12gQ

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.https://www.twitch.tv/openobservabilityhttps://www.youtube.com/@openobservabilitytalks  


Show Notes:

00:00 - show intro

01:03 - episode and guests intro

04:02 - trends and advancements in the Profiling space

05:42 - from cost and performance into broader observability 

11:27 - turning profile data into metrics

12:45 - runtime vs. full host profilers and eBPF use

18:44 - pprof JFR and other existing profile standards

21:19 - profile visualizations - from flame graphs to timeline view 

22:37 - entrepreneur PoV on the profiling market

26:54 - OpenTelemetry adds profiles as a new signal

32:22 - OTel choosing a pprof extended standard  

39:06 - discrete events vs. pre-aggregated data

41:09 - use cases for processing profiling data

44:19 - OTel Profiles reference implementation 

49:11 - latest milestone and roadmap

54:44 - who’s involved in OTel Profiles

56:41 - how to follow OTel Profiles and the guests

59:34 - March community events and conferences

1:00:38 - Falco and CloudEvents projects reached CNCF graduation  

1:01:59 - Prometheus and Linkerd latest releases

1:03:29 - Netflix open-sources bpftop CLI for eBPF app performance monitoring

1:05:15 - show outro


Resources:


Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠


Dotan Horovits

============

Twitter: https://twitter.com/horovits 

LinkedIn: https://www.linkedin.com/in/horovits/ 

Mastodon: @horovits@fosstodon.org


Felix Geisendörfer

===============

Twitter: https://twitter.com/felixge 

LinkedIn: https://www.linkedin.com/in/felixg2/ 


Ryan Perry

==========

Twitter: https://twitter.com/rperry_ 

Linkedin: https://www.linkedin.com/in/ryanaperry/ 


Mar 28, 202401:06:45
Decoding .NET8: Unveiling Cloud-Native Observability - OpenObservability Talks S4E09

Decoding .NET8: Unveiling Cloud-Native Observability - OpenObservability Talks S4E09

The .NET programming language is taking cloud native deployment and observability seriously, and most notably with the recent announcement of .NET Aspire stack unveiled at the recent .NET Conf 2023. 

In this episode, we reviewed the open source maintainers’ journey to making .NET a "by default, out of the box observable platform", as ASP.NET Core creator David Fowler put it. David was this episode’s guest, and with him we dived into .NET Aspire and how it simplifies the complexities of cloud app development with capabilities around service discovery, observability, and resilience. We discussed the local developer experience, the path to developer observability, and what we can expect from the upcoming GA release of .NET8.

David Fowler has been at Microsoft for 15 years working on developer frameworks and tools in the .NET space. He's one of the creators of several popular OSS frameworks and tools such as NuGet, SignalR and ASP.NET Core, and also architected the Azure SignalR Service. Originally from Barbados, he's an avid open-source advocate and developer currently focused on simplifying developer experiences in the microservice space. 


The episode was live-streamed on 21 February 2024 and the video is available at https://www.youtube.com/watch?v=Ll1T9Zs7jUo

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.youtube.com/@openobservabilitytalks  

https://www.twitch.tv/openobservability⁠

Show Notes:

00:10 - episode and guest intro

04:03 - what .NET used to be like for developing cloud-native

15:10 - out-of-the-box observability in .NET

21:05 - .NET aligning with OpenTelemetry

26:40 - what’s .NET Aspire

32:37 - existing .NET components are part of Aspire

37:46 - developing an observability UI as part of Aspire

43:24 - how to transition containerized apps from dev to prod 

48:48 - the relationship between Aspire to Dapr and Radius

53:31 - Aspire roadmap to GA

57:13 - where to follow Aspire and David Fowler

59:13 - K8sgpt accepted to CNCF as a sandbox project

59:56 - Strimzi reaches CNCF incubation with Kafka on Kubernetes

1:00:40 - OpenFeature becomes a CNCF incubating project

1:03:03 - Broadcom kills free ESXi and other VMware restructuring


Resources:

.NET Aspire GitHub repo: https://github.com/dotnet/aspire 

.NET Aspire Preview 3: https://learn.microsoft.com/en-us/dotnet/aspire/whats-new/preview-3 

Instrumenting C# .NET apps with OpenTelemetry: https://logz.io/blog/csharp-dotnet-opentelemetry-instrumentation/?utm_source=devrel&utm_medium=devrel

OpenTelemetry beginner’s guide: https://logz.io/learn/opentelemetry-guide/?utm_source=devrel&utm_medium=devrel

K8sgpt accepted to CNCF sandbox: https://www.linkedin.com/posts/horovits_genai-sre-kubernetes-activity-7158185284289888256-0KuZ

Strimzi reaches CNCF incubation: https://www.cncf.io/blog/2024/02/08/strimzi-joins-the-cncf-incubator/

OpenFeature reaches CNCF incubation: https://www.cncf.io/blog/2023/12/19/openfeature-becomes-a-cncf-incubating-project/

Broadcom decided to kill the free edition of ESXi and other VMware restructuring: https://horovits.medium.com/0aea7efafb47


Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠


Dotan Horovits ============ Twitter: https://twitter.com/horovits LinkedIn: https://www.linkedin.com/in/horovits/ Mastodon: https://fosstodon.org/@horovits David Fowler ========== Twitter: https://twitter.com/davidfowl LinkedIn: https://www.linkedin.com/in/davidfowl Mastodon: https://hachyderm.io/@davidfowl

Feb 29, 202401:05:30
Scaling Platform Engineering: Shopify’s Blueprint - OpenObservability Talks S4E08

Scaling Platform Engineering: Shopify’s Blueprint - OpenObservability Talks S4E08

In this episode, join us as we delve into the intricate world of Platform Engineering with Aparna Subramanian, Director of Production Engineering at Shopify. Discover how Shopify, a powerhouse in e-commerce, masters the art of scaling platform engineering. Gain invaluable insights into their strategies, innovations, and lessons learned while navigating the complexities of sustaining and evolving a robust infrastructure to support millions, even through special peak events like Black Friday and Cyber Monday. If you're keen on understanding the backbone of a thriving online platform, don’t miss out on this episode.

Aparna started her career as a Software Engineer and has spent most part of her almost two decades of technology experience specializing in Infrastructure and Data Platforms. In her current role she leads Shopify’s Cloud Native Production Platform.

Previously, she was Director of Engineering at VMware where she was a founding member of Tanzu on vSphere, a Kubernetes Platform for the hybrid cloud. She also serves as co-chair of the “CNCF End User Developer Experience” SIG and as member of the CNCF End user technical advisory board.

The episode was live-streamed on 11 January 2024 and the video is available at https://www.youtube.com/watch?v=6ShtsTTUizI

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.youtube.com/@openobservabilitytalks  

⁠https://www.twitch.tv/openobservability


Show Notes:

00:00 - Show intro & 2023 stats

01:49 - Episode and guest intro

04:15 - Shopify’s scale

06:09 - Shopify’s journey to Platform Engineering

08:56 - Shopify’s platform structure

11:49 - division of responsibility

13:51 - golden path vs flexibility

17:58 - balancing flexibility and abstraction

19:56 - platform group structure

23:28 - handling load spikes

28:55 - FinOps in Platform Engineering

38:38 - avoiding silos and the cultural aspect

41:13 - CNCF end-user SIG and community challenges

49:24 - KubeCon Paris and guest contact 

51:03 - OpenTofu reached GA

53:33 - Isovalent acquired by Cisco

55:00 - year-end summary articles

57:07 - .NET Aspire released preview2

58:58 - Episode and show outro


Resources:

Shopify Engineering Blog https://shopify.engineering/

Performance wins at Shopify: https://www.shopify.com/news/performance%F0%9F%91%86-complexity%F0%9F%91%87-killer-updates-from-shopify-engineering

CNCF End User SIG https://github.com/cncf/enduser-public

OpenTofu has reached GA https://logz.io/blog/terraform-is-no-longer-open-source-is-opentofu-opentf-the-successor/?utm_source=devrel&utm_medium=devrel

Observability in 2024: https://thenewstack.io/observability-in-2024-more-opentelemetry-less-confusion/

OpenTelemetry in 2024: https://www.apmdigest.com/2024-application-performance-management-apm-predictions-4

.NET Aspire preview2: https://devblogs.microsoft.com/dotnet/announcing-dotnet-aspire-preview-2/ 


Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠


Dotan Horovits

============

Twitter: @horovits

LinkedIn: in/horovits

Mastodon: @horovits@fosstodon


Aparna Subramanian

=================

Twitter: @aparnastweets

LinkedIn: https://www.linkedin.com/in/subramanianaparna/ 


Jan 25, 202401:00:17
Kubernetes and Beyond: A Year-End Reflection with Kelsey Hightower - OpenObservability Talks S4E07

Kubernetes and Beyond: A Year-End Reflection with Kelsey Hightower - OpenObservability Talks S4E07

In this special episode we wrapped up the year 2023 with none other than the cloud-native maestro, Kelsey Hightower! We looked into the highs and lows of the tech landscape, exploring Kelsey's insights on containerization and beyond. Tune in as we unravel the year that was and reflect on what lies ahead for Kubernetes and cloud computing.

Kelsey has been there since the birth of Kubernetes, with his contributions to the project as well as his advocacy for containers and cloud native tech and concepts. Join us to conclude 2023 with a look above the clouds.

The episode was live-streamed on 5 December 2023 and the video is available at https://www.youtube.com/watch?v=OVSIUMJxtLk

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.twitch.tv/openobservability

https://www.youtube.com/@openobservabilitytalks  


Show Notes:

00:00 - Show intro

01:00 - Episode and guest intro

02:40 - Highlights of 2023, signs of maturity

05:17 - Standardizing on cloud bills

12:09 - Consensus vs. innovation in tech

14:46 - Evolution of OpenTelemetry and telemetry signals

19:33 - Where AI will help DevOps and Observability

25:44 - Where is Kubernetes heading in the coming decade

32:42 - Can Kubernetes serve AI/ML workloads

40:37 - CNCF landscape - transparency vs. complexity

49:05 - Evolution of observability

59:03 - Episode and show outro


Resources:

Standardizing on cloud bills with FOCUS open specification: https://horovits.medium.com/6e30069f33a0

How to fix Kubernetes monitoring: https://thenewstack.io/how-to-fix-kubernetes-monitoring/


Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠

Dec 21, 202301:01:05
KubeCon NA Highlights and Istio Spotlight with Lin Sun - OpenObservability Talks S4E06

KubeCon NA Highlights and Istio Spotlight with Lin Sun - OpenObservability Talks S4E06

Have you missed KubeCon North America in Chicago? This one’s for you! In this episode, we explored the latest and greatest highlights from the event that every tech enthusiast is talking about. From cutting-edge innovations to industry insights, we've got the broad spectrum covered. 

But that's not all! We'll also zoomed in on Istio, the popular service mesh open source project that has just recently reached CNCF graduation. Join us as we map out the service mesh universe, and then dive into Istio's galaxy, unraveling its architecture, features, and the roadmap direction with Ambient. And you’ll get to hear it from the Istio authority, Lin Sun.

Lin is the Director of Open Source at Solo.io and a CNCF ambassador. She has worked on the Istio service mesh since the beginning of the project in 2017 and serves on the Istio Steering Committee and Technical Oversight Committee. Previously, she was a Senior Technical Staff Member and Master Inventor at IBM for 15+ years. She is the author of the book "Istio Ambient Explained" and co-author of “Istio Explained”, and has more than 200 patents to her name.

The episode was live-streamed on 15 November 2023 and the video is available at https://www.youtube.com/watch?v=bxnDH6LH-cA

You can read the recap post: https://logz.io/blog/kubecon-na-2023-recap/?utm_source=devrel&utm_medium=devrel

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.https://www.twitch.tv/openobservabilityhttps://www.youtube.com/@openobservabilitytalks  


Show Notes:

01:27 - Episode and guest intro

06:34 - KubeCon Highlights: Fluent Bit

09:16 - OpenTelemetry Logging, OTLP is GA

12:53 - OpenTelemetry project journey report

13:43 - WASM Day and Istio Day updates

16:18 - Keynote: the future of Kubernetes

18:51 -Crossplane latest release v1.14 

19:24 - Kyverno supports non-Kubernetes workloads

20:12 - Vitess 18 is now GA

20:43 - AI is nascent in CNCF

22:56 - CNCF’s GitOps microsurvey 

23:56 - eBPF documentary released

27:08 - Service Mesh architecture and landscape

31:36 - Envoy proxy 

33:48 - maturity of the projects

39:36 - Istio unique value proposition and adoption

43:55 - Kubernetes released native sidecar support

47:02 - The GAMMA initiative in Kubernetes Gateway API 

50:04 - Istio updates: Ambient, multi-claster, Gateway API GA impl. For N-S 

53:40 - CNCF Training & Certification Launch Istio Certification

54:56 - Istio roadmap

56:50 - how to follow Istio and Lin Sun and episode wrapup


Resources:

KubeCon Updates:

https://www.cncf.io/blog/2023/11/07/opentelemetry-at-kubecon-cloudnativecon-north-america-2023-update/

https://opentelemetry.io/blog/2023/http-conventions-declared-stable/

https://www.cncf.io/reports/opentelemetry-project-journey-report/ 

https://blog.crossplane.io/crossplane-v1-14/  

https://www.cncf.io/blog/2023/11/06/kyverno-expands-beyond-kubernetes/ 

https://planetscale.com/blog/announcing-vitess-18 

https://www.cncf.io/blog/2023/11/07/cncf-gitops-microsurvey-learning-on-the-job-as-gitops-goes-mainstream/ 


Istio Spotlight:

https://istio.io/latest/blog/2023/native-sidecars/

https://istio.io/latest/blog/2022/introducing-ambient-mesh/

https://gateway-api.sigs.k8s.io/concepts/gamma/

https://www.cncf.io/announcements/2023/07/12/cloud-native-computing-foundation-reaffirms-istio-maturity-with-project-graduation/ 

https://istio.io/latest/get-involved/

https://training.linuxfoundation.org/blog/istio-certification/ 

https://www.oreilly.com/library/view/istio-ambient-explained/9781098142698/


Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠



Nov 30, 202301:00:12
PromCon Recap: Unveiling Perses and Prometheus Ecosystem Updates - OpenObservability Talks S4E05

PromCon Recap: Unveiling Perses and Prometheus Ecosystem Updates - OpenObservability Talks S4E05

PromCon, the flagship yearly event of the Prometheus community, took place in Berlin 28-29 September 2023, and we’re here to bring you the highlights from the Prometheus ecosystem, including the pivotal decision on Prometheus 3.0! Brace yourselves for some exciting announcements! We also delved into the latest addition to the ecosystem, Perses project, which promises to revolutionize the world of dashboard visualization and monitoring. This new open source project, now part of the Linux Foundation, aims to become the GitOps-friendly standard dashboard visualization tool for Prometheus and other data sources. On this episode I hosted Augustin Husson, Prometheus maintainer and the creator of the Perses project, at the heels of his PromCon announcement of the Perses release. Augustin is also principal engineer at Amadeus, a technology vendor for travel agencies. Augustin joined Amadeus to create a new internal monitoring system based on Prometheus, and he will also share his end-user journey and insights.

The episode was live-streamed on 4 October 2023 and the video is available at https://www.youtube.com/watch?v=MzQZagfgIKk

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.youtube.com/@openobservabilitytalks  

https://www.twitch.tv/openobservability


Show Notes:

00:00 - show, episode and guest intro

05:46 - OpenTelemetry support in Prometheus

11:02 - Green IT use case with Prometheus

14:12 - scrape sharding support in Prometheus operator 

19:33 - scaling out Alerts and alert sharding

24:45 - Windows Exporter is released

27:50 - revamping the Prometheus UI with React  

30:48 - Prometheus 3.0 and DevDay updates

41:04 - Perses project origins at Amadeus 

47:10 - Perses joining open source foundation

49:58 - embedding Perses in Red Hat OpenShift and in Chronosphere 

54:05 - Perses current release 

59:32 - Perses roadmap

1:03:11 - Perses joining the Linux Foundation and the CNCF

1:07:47 - how to get involved in Perses

1:10:03 - episode outro


Resources:

Perses on GitHub: https://github.com/perses/perses The CoreDash Project: https://github.com/coredashio/community Perses overview talk at PromCon 2023: https://promcon.io/2023-berlin/talks/... Prometheus support for OpenTelemetry Metrics in OTLP: https://horovits.medium.com/83f85878e46a


Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠


Dotan Horovits

============

Twitter: @horovits

LinkedIn: in/horovits

Mastodon: @horovits@fosstodon


Augustin Husson =============== Twitter: https://twitter.com/nexucis LinkedIn: https://fr.linkedin.com/in/augustin-husson-69a050a1 Mastodon: https://hachyderm.io/@nexucis

Oct 31, 202301:11:07
Continuous Observability: Shedding Light on CI/CD Pipelines - OpenObservability Talks S4E02

Continuous Observability: Shedding Light on CI/CD Pipelines - OpenObservability Talks S4E02

DevOps is not just about operating the software in production, but also about releasing that software to production. Well-functioning CI/CD pipelines are critical for the business, and this calls for quality observability, to handle broken and flaky pipeline runs effectively. On this episode I hosted Oleg Nenashev, a core maintainer and board member in the Jenkins project, as well as a TOC member in the Continuous Delivery Foundation (CDF). Oleg is a community builder, open source advocate and consultant, now at WireMock Inc. He is also a CDF and CNCF ambassador. We discussed CI/CD, observability, the prominent open source projects and foundations, as well as a new proposal for extending OpenTelemetry to natively support CI/CD observability use cases.

The episode was live-streamed on 10 July 2023 and the video is available at https://www.youtube.com/watch?v=FEbyddZFNeo

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.twitch.tv/openobservability

https://www.youtube.com/@openobservabilitytalks  


Show Notes:

00:00 - show intro

01:00 - episode and guest intro

10:08 - what’s new in Jenkins

15:46 - is Jenkins cloud-native?

16:52 - understanding the CI/CD landscape

21:54 - updates from the Continuous Delivery Foundation

27:00 - CI/CD observability support in OpenTelemetry

40:31 - observability with Backstage IDP open source 

47:47 - how to contact Oleg

48:51 - State of Continuous Delivery report

52:32 - OTLP 1.0 release and other open standards updates

54:32 - KubeCon will hold a dev-centric event for the first time

55:55 - Jaeger 1.47 is out

57:30 - DevOps Pulse survey insights

58:55 - outro


Resources:

Observability Has a Complexity Problem (APM Digest)

OTLP 1.0 is out

Open Standards in observability: updates from KubeCon

CI/CD observability support in OpenTelemetry proposal

State of Continuous Delivery 2023 report

Socials:


Dotan Horovits

============

Twitter: @horovits

LinkedIn: in/horovits

Mastodon: @horovits@fosstodon


Oleg Nenashev

===============

Twitter: @oleg_nenashev

LinkedIn: https://www.linkedin.com/in/onenashev/

Mastodon: @asciidwarf@fosstodon


Oct 31, 202301:00:30
Terraform is no longer open source. Is OpenTofu the successor? - OpenObservability Talks S4E04

Terraform is no longer open source. Is OpenTofu the successor? - OpenObservability Talks S4E04

Terraform is no longer open source. This is the news we got last month (August 2023), when HashiCorp announced its decision to relicense its open source tools, including Terraform, Vault, Packer, Consul, Vagrant and others, into Business Source License 1.1. The community, led by active Terraform-based vendors, gathered up to create a fork of Terraform to keep it open. The result is OpenTofu (originally called OpenTF), whose manifesto already has tens of thousands of stars on GitHub, less than a month out. Only a month old, engineers are hard at work to establish the first release of OpenTofu, as well as its foundational backbone. In this month’s episode I covered these significant events that shake our industry and the DevOps world. I was joined by Omry Hay, co-founder and CTO of env0. env0 provides an automation solution based on Terraform, and is one of the creators of OpenTofu and a member of the project’s steering committee. Omry also shared OpenTofu’s mission and current status, as well as exciting updates, hot off Open Source Summit Europe conference taking place these days, in which OpenTofu has officially joined The Linux Foundation. Omry has been a software engineer and engineering manager for the last 16 years, working at companies like eToro, Fiverr and Proofpoint. As CTO of env0, he leads the R&D and Product departments.

The episode was live-streamed on 18 September 2023 and the video is available at https://www.youtube.com/watch?v=5QdUs9VKq5g

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

⁠https://www.youtube.com/@openobservabilitytalks⁠  

https://www.twitch.tv/openobservability


Show Notes:

00:00 - show intro 00:56 - episode and guest intro 02:45 - HashiCorp’s relicensing announcement 04:58 - what the relicensing means for users 14:50 - implications on the Terraform ecosystem 24:55 - HCL language for IaC 28:36 - what does the new license mean? 32:13 - Terms of service changed for Terraform Registry 36:08 - forking Terraform and starting OpenTF/OpenTofu 41:08 - how many engineers work on OpenTofu 42:18 - joining the Linux Foundation and renaming OpenTofu 48.50 - OpenTofu release and Terraform compatibility 56:54 - roadmap for OpenTofu 59:00 - how to get touch with the community and Omry 64.30 - The OSI Approved Licenses database is available 65:28 - Red Hat changed the CentOS release process

Resources:

HashiCorp relicensing announcement: https://www.hashicorp.com/blog/hashicorp-adopts-business-source-licenseOpenTofu project: https://opentofu.org/ The Linux Foundation announces OpenTofu: https://www.linuxfoundation.org/press/announcing-opentofu Red Hat changed the CentOS release process: https://www.redhat.com/en/blog/furthering-evolution-centos-streamCNCF’s guidelines for using source-available dependencies in its OSS projects: https://github.com/cncf/foundation/blob/main/source-available-recommendations.md#recommendations checklist for safely using and choosing open source tools: https://medium.com/@horovits/when-your-open-source-turns-to-the-dark-side-331d83f182c

Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠


Dotan Horovits

============

Twitter: @horovits

LinkedIn: in/horovits

Mastodon: @horovits@fosstodon

Omry Hay ======== Twitter: https://twitter.com/omryhay LinkedIn: https://www.linkedin.com/in/omryhay/

Sep 21, 202301:09:42
What's New with Fluentd & Fluent Bit - OpenObservability Talks S4E03

What's New with Fluentd & Fluent Bit - OpenObservability Talks S4E03

Fluentd and Fluent Bit are two highly popular open source projects for data collection and log forwarding in the realm of observability. Fluentd's flexibility and scalability have led to seamless integration with diverse applications and data sources, while Fluent Bit's lightweight and efficient log forwarding have made it a preferred choice for modern observability pipelines. But Fluent Bit can process more than just logs. The recent release of Fluent Bit v2 added major new integrations with OpenTelemetry, Prometheus and more, as well as extensibility with WebAssembly plugins.

On this episode I hosted Eduardo Silva Pereira, one of Fluentd project maintainers and creator of Fluent Bit. He also is the founder of Calyptia, the Fluent company. Eduardo shared with us the latest updates of Fluentd and Fluent Bit, as well as valuable insights into the future roadmap of these projects. The episode was live-streamed on 9 August 2023 and the video is available at https://youtu.be/V02Ctv0Rtg8

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.youtube.com/@openobservabilitytalks  

⁠https://www.twitch.tv/openobservability⁠


Show Notes:

02:11 - Fluentd and Fluent Bit background

09:51 - Should I choose FluentD or Fluent Bit?

13:26 - developing an active engaged OSS community

17:18 - enterprise needs and building commercial offering with Calyptia

19:54 - Fluent Bit v2 updates

29:22 - plugins, filters and processors in Fluent Bit

38:23 - A sneak peak into the planned announcements for KubeCon Chicago

44:16 - where to follow the community and Eduardo

47:43 - Prometheus now supports OTLP

48:57 - PromCon will take place in Berlin, 28-29 Sept.

50:11 - OpenTelemetry Semantic Conventions is separated from the Specification

53:38 - New in Kubernetes 1.27: Query node logs using the kubelet API

54:43 - Kelemetry: global tracing for Kubernetes control plane


Resources:

Resources

=========

Fluent Bit: https://fluentbit.io/

FluentD: https://www.fluentd.org/

Prometheus supports OTLP: https://horovits.medium.com/83f85878e46a

PromCon '23: https://promcon.io/2023-berlin/

Query node logs using the kubelet API: https://logz.io/blog/a-practical-guide-to-kubernetes-logging/#Kubernetes_1_27

Kelemetry project: https://www.cncf.io/blog/2023/07/27/kelemetry-global-tracing-for-kubernetes-control-plane/


Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠

Dotan Horovits ============ Twitter: https://twitter.com/horovits LinkedIn: https://www.linkedin.com/in/horovits/ Mastodon: @horovits@fosstodon.org Eduardo Silva Pereira =============== Twitter: https://twitter.com/edsiper LinkedIn: https://www.linkedin.com/in/edsiper

Aug 31, 202358:38
​​eBay’s Journey to Planet-Scale Observability with Open Source - OpenObservability Talks S4E01

​​eBay’s Journey to Planet-Scale Observability with Open Source - OpenObservability Talks S4E01

eBay is a high scale end user of open source observability. Collecting telemetry from millions of endpoints, and running thousands of queries per second, presents serious scaling challenges. eBay has chosen to use an open source stack to meet those challenges. On this episode I hosted Vijay Samuel, Observability Architect at eBay, to hear about the challenges eBay faced in monitoring large Kubernetes installations. We discussed why eBay chose the open source stack, and the strategic decision behind eBay's migration from Elastic Beats to OpenTelemetry, and the remarkable experiences they had while deploying massively large scale telemetry installations using OpenTelemetry and Prometheus. We also discussed a fascinating new proposal for a query language standard for observability, which eBay spearheads together with Netflix under the auspices of the CNCF’s TAG Observability (the CNCF’s technical advisory group for observability).

The episode was live-streamed on 8 June 2023 and the video is available at https://www.youtube.com/watch?v=-UsU3nRglhA

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.twitch.tv/openobservability

https://www.youtube.com/@openobservabilitytalks  


Show Notes:

3rd anniversary

eBay monitoring solution 

planet scale at eBay in numbers

distributed tracing at eBay

migrating from Elastic Beats to OpenTelemetry

why eBay chose open source 

open-sourcing eBay’s metrics store platform?

scaling Prometheus

ramping up tracing, from backend to frontend and mobile

running OpenTelemetry Collector at scale

Query Language Standardization for Observability under the CNCF


Resources:

New CNCF working group for Observability Query Language Standardization: https://github.com/cncf/tag-observability/blob/main/working-groups/query-standardization.md Why and How eBay Pivoted to OpenTelemetry: https://tech.ebayinc.com/engineering/why-and-how-ebay-pivoted-to-opentelemetry/


Socials:


Dotan Horovits

============

Twitter: @horovits

LinkedIn: in/horovits

Mastodon: @horovits@fosstodon


Vijay Samuel

==========

Twitter: @vjsamuel_

LinkedIn: vjsamuel


Jun 29, 202301:00:21
From Spotify to Open Source: The Backstory of Backstage - OpenObservability Talks S3E12

From Spotify to Open Source: The Backstory of Backstage - OpenObservability Talks S3E12

With over 1,000 companies using it and 21.5K+ stars on GitHub, the Backstage open source project is quickly becoming a go-to tool for managing developer infrastructure. In this episode, I’ll sit down with Lee Mills, a Senior Engineering Manager from Spotify's Backstage project, to learn more about how the open source platform is revolutionizing the developer experience and how it aligns with the growing Internal Developer Platform (IDP) space. We’ll discuss the need that drove Spotify to build this internal tool, about the decision and journey to open source it and donate it to the CNCF (Cloud Native Computing Foundation). We’ll also discuss the new commercial plugins, what’s coming up on the roadmap, and much more. 

Lee has worked as an engineering manager for the past 12 years, from Amazon to Spotify and everything in between. Prior to that Lee had a mixed background working in academia as a lecturer, worked as a Boom Op for television in the UK and lived in more places than he can count.

The episode was live-streamed on 17 May 2023 and the video is available at https://youtu.be/W3c4YJ71BOQ

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.https://www.twitch.tv/openobservabilityhttps://www.youtube.com/@openobservabilitytalks  


Show Notes:

01:44 Spotify Engineering internal need for Backstage

07:09 Backstage and Platform Engineering

10:33 Backstage components: Templates, Catalogue and Plugins

15:31 Why Spotify open-sourced Backstage

21:45 from Spotify’s internal Hack Week to a viral growth

25:19 a perfect community contributed feature in 24 hours

28:48 new Backstages use cases for Banking and healthcare

30:54 Spotify released commercial plugins for Backstage

36:51 How to follow Backstage discussion and Lee Mills

39:08 KubeCon Europe updates

40:51 Prometheus remote-write is standardized

41:32 Sunsetting OpenCensus

42:31 ECS to merge with OpenTelemetry specification

43:22 progress to stabilize Logs in OpenTelemetry

45:02 MicroProfile v6.0 support OpenTelemetry Traces

45:40 Grafana 9.5 release


Resources:

Backstage website: https://backstage.io 

Kubecon Europe 2023 talk: The State of Backstage in 2023: https://youtu.be/vskefrlvocE 

Kubecon Europe 2023 talk: Lunar building an internal compliance platform using Backstage: https://youtu.be/6T3Mf6pdg7E

Backstage accepted as CNCF incubation project: https://www.cncf.io/blog/2022/03/15/backstage-project-joins-the-cncf-incubator/


Socials:


Dotan Horovits

============

Twitter: @horovits

LinkedIn: in/horovits

Mastodon: @horovits@fosstodon


Lee Mills =============== LinkedIn: https://www.linkedin.com/in/codetoy Mastodon: @leem@mastodon.social

May 31, 202347:19
Live from KubeCon: Insider Insights with CNCF's Head of Ecosystem - OpenObservability Talks S3E11

Live from KubeCon: Insider Insights with CNCF's Head of Ecosystem - OpenObservability Talks S3E11

This is a special episode, live from the KubeCon show floor in Amsterdam. Join us to hear the hot updates from KubeCon + CloudNativeCon Europe 2023 as well as insider insights from the CNCF’s head of ecosystem, Taylor Dolezal. Taylor works on infrastructure tools that enable innovation. He specializes in Kubernetes, Terraform, public clouds, and distributed systems. Taylor will also deliver the opening keynote on the upcoming KubeCon EU in Amsterdam. The episode was live-streamed on 20 April 2023 and the video is available at https://www.youtube.com/live/a9D5p0SaKL8?feature=share

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.twitch.tv/openobservability

https://www.youtube.com/@openobservabilitytalks  

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/


Show Notes:

  • KubeCon EU 2023 stats
  • KubeCon EU 2024 plan
  • CTO Summit EU 2023 focus on FinOps best practices
  • End user challenges
  • Getting end users involved in the OSS
  • Status of end user cloud native maturity
  • Unified Query Language new working group
  • This KubeCon's hallway topics and Observability co-lo event
  • CTO summit report and community feedback


Resources:

Socials:


Dotan Horovits

============

Twitter: @horovits

LinkedIn: https://www.linkedin.com/in/horovits/ 

Mastodon: @horovits@fosstodon


Taylor Dolezal =========== Twitter: @onlydole LinkedIn: https://www.linkedin.com/in/onlydole/


Apr 27, 202327:31
Cloud Native Unplugged: A Fireside Chat with CNCF's CTO - OpenObservability Talks S3E10

Cloud Native Unplugged: A Fireside Chat with CNCF's CTO - OpenObservability Talks S3E10

The Cloud Native Computing Foundation (CNCF) is the home of the most prominent open source projects used today, such as Kubernetes, Prometheus, ArgoCD, Linkerd and more. These projects fuel today’s cloud native architectures and software release pipelines. With its immense growth, it has become difficult to keep tabs on the hundreds of new and evolving projects and specifications, the different working groups and technical advisory groups, the different community forums and events, and to see where it’s all heading.

I invited Chris Aniszczyk, CTO of the CNCF, to join me on this episode, to help us understand the CNCF landscape and evolution. We will also discuss the trends in observability and in the open source realm in general. Chris also has some interesting predictions to share. 

Chris Aniszczyk is an open source executive and engineer with a passion for building a better world through open collaboration. He's currently a CTO at the Linux Foundation focused on developer relations and running the Open Container Initiative (OCI) / Cloud Native Computing Foundation (CNCF). Furthermore, he's a partner at Capital Factory where he focuses on mentoring, advising and investing in open source and infrastructure focused startups.

At Twitter, he created their open source program and led their open source efforts. For many years he served on the Eclipse Foundation's Board of Directors representing the committer community and the Java Community Process (JCP) Executive Committee. In a previous life, he bootstrapped a consulting company, made many mistakes, lead and hacked on many eclipse.org and Linux related projects.

The episode was live-streamed on 15 March 2023 and the video is available at https://www.youtube.com/live/lMUFGmNploc

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.

https://www.twitch.tv/openobservability

https://www.youtube.com/@openobservabilitytalks  

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/


Show Notes:

  • Day in a life of CNCF CTO
  • open source sustainability
  • how to navigate the CNCF landscape
  • how to get started with cloud native
  • is Kubernetes spreading too broad to lose focus?
  • OpenTelemetry project journey report sneak peak
  • open observability stack convergence
  • OpenFeature feature flagging OSS
  • CNCF investing in more regional activity
  • CNCF investing in security
  • relicensing and OSS citizenship issues
  • CNCF project health dashboard
  • KubeCon sneak peak

Resources:

Socials:


Dotan Horovits

============

Twitter: https://twitter.com/horovits 

LinkedIn: https://www.linkedin.com/in/horovits/ 

Mastodon: @horovits@fosstodon


Chris Aniszczyk

===============

Twitter: @cra

LinkedIn: https://www.linkedin.com/in/caniszczyk/

Mastodon: @cra@macaw.social


Mar 30, 202301:05:01
FinOps Observability: Monitoring Kubernetes Cost with OpenCost - OpenObservability Talks S3E09

FinOps Observability: Monitoring Kubernetes Cost with OpenCost - OpenObservability Talks S3E09

Many organizations struggle in understanding and monitoring the costs of their Kubernetes workloads, cloud infrastructure and cloud native applications. Moreover, different cloud providers use different conventions, which makes it difficult to compare across vendors and to monitor cost in multi-cloud environments. The lack of cost observability and vendor-agnostic FinOps standardization can become a critical business challenge.  

OpenCost is a vendor-neutral open source project for measuring and allocating infrastructure and container costs. It’s built for Kubernetes cost monitoring to power real-time cost monitoring, showback, and chargeback, across on-premises Kubernetes as well as cloud managed offering.  

I discussed this topic with Matt Ray, Senior Community Manager for the OpenCost project. Matt  has been active in Open Source and DevOps communities for over two decades and has spoken at and helped organize many conferences and meetups. He is currently the Senior Community Manager at Kubecost for the CNCF Sandbox Project OpenCost. Matt also co-hosts the Software Defined Talk podcast.

The episode was live-streamed on 14 February 2023 and the video is available at https://www.youtube.com/watch?v=mhqXQV2jsxo

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/@openobservabilitytalks

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

Show Notes:

  • FinOps and the FinOps Foundation
  • Relevant stakeholders
  • Understanding your public cloud bill
  • How is Kubernetes spend different
  • OpenCost project overview
  • OpenCost roadmap and ecosystem
  • How to join OpenCost convo
  • News and updates

Resources:

  • https://www.cncf.io/blog/2022/12/06/opencost-a-new-cncf-sandbox-project-for-real-time-kubernetes-cost-monitoring/ 
  • https://www.opencost.io/
  • https://github.com/opencost/opencost 
  • https://logz.io/blog/finops-distributed-tracing/

Socials:


Dotan Horovits

============

Twitter: horovits

LinkedIn: https://www.linkedin.com/in/horovits/

Mastodon: @horovits@fosstodon


Matt Ray

===============

Twitter: mattray

LinkedIn: https://www.linkedin.com/in/mhray/

Mastodon: @mattray@mastodon.social

Feb 23, 202301:01:21
Is Kubernetes Monitoring Flawed? - OpenObservability Talks S3E08

Is Kubernetes Monitoring Flawed? - OpenObservability Talks S3E08

A 3-node Kubernetes cluster with Prometheus will ship around 40k active series by default! Do we really need all that data? The current state of Kubernetes open source monitoring is in need of improvement. High churn rate of pod metrics, proliferation of metrics with low usage, and configuration complexity are some of the issues that need to be addressed.

I discussed this topic with Aliaksandr Valialkin, CTO at VictoriaMetrics and creator of the open source project. We discussed the common problems, as well as directions and best practices to overcome some of these complexities as individuals and as a community. We also discussed VictoriaMetrics open source project and how it addresses some of these challenges.

Aliaksandr a Golang engineer, who likes writing simple and performant code and creating easy-to-use programs. Sometimes these hard-to-match requirements work together, like in the VictoriaMetrics case.

The episode was live-streamed on 24 January 2023 and the video is available at https://www.youtube.com/live/Z-58C8HFGb8

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/@openobservabilitytalks

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

Show Notes:

  • monitoring microservice system, app and communications
  • high churn rate for pod metrics
  • Kubernetes produces too many metrics by defaults, most of which are unused
  • recommended listing of metrics
  • removing unused metric labels to reduce cardinality
  • Prometheus native (exponential buckets) historgrams
  • Configuration complexity with multiple deployments
  • OpenTelemetry and OpenMetrics open specifications
  • collecting system metrics and application metrics uniformly
  • VictoriaMetrics essentials
  • VictoriaMetrics extensions beyond Prometheus
  • a full stack monitoring collection, analysis and alerting
  • how to join the VictoriaMetrics community
  • industry update: 2023 cloud native predictions post by CNCF CTO

Resources:

  • Why Prometheus cannot query remote storage in an expected way via remote_read protocol - https://github.com/prometheus/prometheus/issues/4456 
  • VictoriaMetrics scaling to 100 million metrics per second https://www.youtube.com/watch?v=xfed9_Q0_qU
  • https://victoriametrics.com/
  • https://github.com/VictoriaMetrics/VictoriaMetrics
  • https://docs.victoriametrics.com/#community-and-contributions

Socials:

Dotan Horovits 

Twitter: https://twitter.com/horovits  
LinkedIn: https://www.linkedin.com/in/horovits/  


Aliaksandr Valialkin

Twitter: https://twitter.com/valyala
LinkedIn: https://www.linkedin.com/in/valyala/

Jan 26, 202301:00:32
What's new in the Prometheus ecosystem? - OpenObservability Talks S3E07

What's new in the Prometheus ecosystem? - OpenObservability Talks S3E07

So much has been going on with the Prometheus project and its ecosystem, that it’s time to have a proper catch up. And there’s no better person to walk us through it than Julien Pivotto, who debriefed the community last month at PromCon.

Julien Pivotto is a maintainer of Prometheus, the open source monitoring and alerting solution. He is the co-founder of the company O11y, that provides premium support for open source observability tools such as Prometheus, Thanos and Grafana.

The episode was live-streamed on 19 December 2022 and the video is available at https://www.youtube.com/watch?v=Vui4EgveUxg

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/@openobservabilitytalks

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

Show Notes:

  • Prometheus is 10 years old
  • Prometheus mission statement and directions
  • New Agent Mode for work with external backends
  • Service Discovery ecosystem, plugins and updates
  • Time series database basics and updates
  • New support for native histograms
  • Examplars in Prometheus to correlate metrics to traces
  • PromQL query language updates
  • PromLens contribution to Prometheus
  • Prometheus UI updates
  • Visualization options: Grafana and Perses 
  • Alertmanager updates
  • Windows exporter, MySQL and other new exporters
  • Long term support for Prometheus project
  • Thanos, Cortex, Mimir - ecosystem update
  • Prometheus community  

Resources:

Socials:

Dec 26, 202201:04:27
Meta’s data driven approach to observability - OpenObservability Talks S3E06

Meta’s data driven approach to observability - OpenObservability Talks S3E06

At Meta (Facebook, Instragram et al) everything is data, and data driven approach is the rule, from product to engineering, from HR to finance. This is also how the team at Meta treats observability. Let’s see how we treat observability as a data analytics problem, and what you can implement, even if you’re not a hyperscaler.

On this episode I’ll host David Ostrovsky from Meta. David is a software developer with over 20 years of industry experience, speaker, trainer, blogger and co-author of “Pro Couchbase Server”. He specializes in large-scale distributed system architecture.

The episode was live-streamed on 7 November 2022 and the video is available at https://youtu.be/1l0HKUDoX4Q

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

Show Notes:

  • Addressing business observability
  • Machine learning and predictions in observability
  • From the business inwards, using SLOs
  • Accelerate engineering quality with developer observability
  • Organizational and communications aspects of high scale observability
  • Actionable observability
  • How small-medium size orgs can achieve a similar effect
  • OpenTelemetry demo is GA
  • PromLens is open sourced and contributed to Prometheus

Resources:

Socials:

Nov 23, 202201:00:20
Platform Engineering: DevOps evolution or a fancy rename? - OpenObservability Talks S3E05

Platform Engineering: DevOps evolution or a fancy rename? - OpenObservability Talks S3E05

Everyone’s talking about Platform Engineering these days. Even Gartner featured it in its Hype Cycle for Software Engineering 2022. But what is Platform Engineering really about? Is it the next stage in the evolution of DevOps? Is it just a fancy rebrand for DevOps or SRE? And how does observability pertain to platform engineering?

On this episode of OpenObservability Talks Horovits hosted George Hantzaras, Director of Cloud Platform Engineering at Citrix. George is a distributed systems expert and a hands-on engineering leader with focus on delivering B2B cloud services at scale. Coming from a DevOps background, he focuses on implementing SRE at enterprise scale working with cloud native technologies. He has been organizing the Athens Cloud Computing Meetup since 2016 and the Athens Hashicorp User Group. Most recently, he has been a speaker at global events like Hashiconf, DeveloperWeek, Voxxed Days, DevNexus and more, focusing on reliability engineering, agile leadership, scaling engineering teams, and entrepreneurship.

The episode was live-streamed on 6 October 2022 and the video is available at https://youtu.be/zJGMUVY6fDM

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

Show Notes:

  • Why Platform Engineering?
  • Different teams employ different stacks. How does Platform Engineering fits in?
  • Platform Engineering at Citrix
  • How does it map to the common DevOps maturity models?
  • Is Platform Engineering different from PaaS?
  • Platform as a product
  • Culture element is central in Platform Engineering
  • Generating buy-in within the product’s engineering
  • Metrics and quantifying the benefits of Platform Engineering
  • Balancing flexibility and simplicity, and striking the right level of abstraction
  • Delimitation between what’s developed by Platform vs. Product teams.
  • Generating Golden Paths for engineering
  • Observability and Platform Engineering

Resources:

Socials:

Oct 27, 202258:15
Where Are My App’s Traces?? Instrumentation in Practice - OpenObservability Talks S3E04

Where Are My App’s Traces?? Instrumentation in Practice - OpenObservability Talks S3E04

Instrumentation is that black magic that makes our application emit traces, logs, metrics or other telemetry. How does it work? What options are available in different programming languages, such as Java, Python and Go? What does OpenTelemetry offer in this domain?  

On this episode of OpenObservability Talks I hosted Eden Federman, Co-Founder & CTO at keyval, a company focused on making observability simpler. Eden is the creator of two open source projects: Odigos and Go automatic instrumentation (now part of OpenTelemetry). Eden is passionate about everything related to observability and performance monitoring. He also created kubectl-flame, a profiler for Kubernetes.

The episode was live-streamed on 22 September 2022 and the video is available at https://youtu.be/VFykWV1mLAI

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

Show Notes:

  • What is instrumentation
  • Manual and Automatic instrumentation
  • Different languages offer different options
  • Java instrumentation capabilities
  • Go instrumentation capabilities
  • Instrumentation when using programming frameworks
  • eBPF use in auto-instrumentation
  • New OpenTelemetry SIG for Go auto-instrumentation
  • Odigos open source project
  • Best practices for instrumentation

Resources:

Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠


Sep 29, 202201:00:26
SigNoz: Open-source observability with Pranay - OpenObservability Talks S3E03
Aug 25, 202256:17
Expensive Observability: The Cardinality Challenge - OpenObservability Talks S3E02

Expensive Observability: The Cardinality Challenge - OpenObservability Talks S3E02

We all collect logs, metrics and perhaps traces and other data types, in support of our observability. But this can get expensive pretty quickly, especially in microservices based systems, in what is commonly known as “the cardinality problem”.  

On this episode of OpenObservability Talks I’ll host Ben Sigelman, co-founder and the GM of Lightstep, to discuss this data problem and how to overcome it. Ben architected Google’s own planet-scale metrics and distributed tracing systems (still in production today), and went on to co-create the open-source OpenTracing and OpenTelemetry projects, both part of the CNCF.

The episode was live-streamed on 12 July 2022 and the video is available at https://youtu.be/gJhzwP-mZ2k

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

Show Notes:

  • The difference between monitoring, observability and APM
  • What comprises the cost of observability
  • How common is the knowledge of cardinality and how to add metrics
  • Controlling cost with sampling, verbosity and retention
  • Lessons from Google’s metrics and tracing systems
  • Using metric rollups and aggregations intelligently
  • Semantic conventions for logs, metrics and traces
  • OpenCost project
  • New research paper by Meta on schema-first approach to application telemetry metadata
  • OTEL code contributions - published stats

Resources:

Socials:


Jul 28, 202201:00:03
OpenTelemetry and the Vision for Unified Open Observability - OpenObservability Talks S3E01

OpenTelemetry and the Vision for Unified Open Observability - OpenObservability Talks S3E01

OpenTelemetry is one of the most fascinating and ambitious open source projects of this era. It’s currently the second most active project in the CNCF (the Cloud Native Computing Foundation), with only Kubernetes being more active. The entire industry is aligning behind this project, including incumbent monitoring vendors that were deeply vested in proprietary and closed-source agents to that end.

In this episode of OpenObservability Talks I’ll host Alolita Sharma to discuss OpenTelemetry, its origins and mission statement, as well as updates hot off the press from the recent KubeCon conference in Valencia about releases and future plans.

Alolita is co-chair of the CNCF Technical Advisory Group for Observability, member of the OpenTelemetry Governance Committee and a board director of the Unicode Consortium. She has served on the boards of the OSI and SFLC.in. Alolita has led engineering teams at Wikipedia, Twitter, PayPal, IBM and AWS. Two decades of doing open source continue to inspire her.

The episode was live-streamed on 15 June 2022 and the video is available at https://youtu.be/IK2TWOzDUBI 

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg


You can read the recap post: https://logz.io/blog/opentelemetry-roadmap-and-latest-updates/?utm_source=devrel&utm_medium=devrel


Show Notes:

  • Hot updates from KubeCon EMEA 2022
  • Alolita Sharma introduction
  • The state of OpenTelemetry
  • When OpenTelemetry Logging is expecting GA
  • The onboarding challenge of instrumentation
  • Client side instrumentation and real user monitoring
  • Adding continuous profiling telemetry to OpenTelemetry
  • Interoperability between OpenTelemetry and Prometheus
  • Challenges in OpenTelemetry and observability
  • Where OpenTelemetry is heading next
  • Jaeger OSS now accept OTLP (OpenTelemetry protocol)

Resources:

  • OpenTelemetry Metrics reaches RC: https://opentelemetry.io/blog/2022/metrics-announcement/
  • OpenTelemetry guide: https://logz.io/learn/opentelemetry-guide/
  • CI/CD Observability: https://horovits.medium.com/fighting-slow-and-flaky-ci-cd-pipelines-starts-with-observability-19da2ac94677
  • Jaeger can now accept OpenTelemetry protocol https://medium.com/jaegertracing/introducing-native-support-for-opentelemetry-in-jaeger-eb661be8183c
  • OTel Community Day summary: http://paulsbruce.io/blog/2022/06/opentelemetry-community-day-austin-2022
  • Contextual Logging in Kubernetes 1.24 https://kubernetes.io/blog/2022/05/25/contextual-logging/ 
  • PolarSignals announced FrostDB https://www.polarsignals.com/blog/posts/2022/05/04/introducing-arcticdb/ 

Socials:

Dotan Horovits

============

Twitter: @horovits

LinkedIn: in/horovits

Mastodon: @horovits@fosstodon


Alolita Sharma

============

Twitter: @alolita

LinkedIn: https://www.linkedin.com/in/alolita/


Jun 28, 202201:00:37
Observability for Developers Demystified - OpenObservability Talks E2E12

Observability for Developers Demystified - OpenObservability Talks E2E12

Developers hate monitoring, but we need it. We need it in many points of the software development lifecycle: before deprecating an API, before launching a new feature, after launching the feature, and more. In fact, monitoring needs can vary much more than the classic Ops monitoring.

In this episode I’ll host Liran Haimovitch to discuss how to determine what developers should be monitoring, the difference between observability for Dev and for Ops, and how observability fits into our current dev tools, dev stack and dev processes.

Liran is the Co-Founder and CTO of Rookout. He’s an Observability and Instrumentation expert with a deep understanding of Java, Python, Node, and C++. Liran has broad experience in cybersecurity and compliance from his past roles. When not coding, you can find Liran hosting his podcast, speaking at conferences, writing about his tech adventures, and trying out the local cuisine when traveling.

The episode was live-streamed on 10 May 2022 and the video is available at https://youtu.be/OaHQp-qnVN0 

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

Have you got an interesting topic you'd like to share in an episode? Reach out to us and submit your proposal at https://openobservability.io/

Show Notes:

  • Which data do we need to collect for our observability
  • How is observability for dev different from ops
  • How does observability fit into dev tool stack
  • Snapshots provide deep-dive telemetry signal
  • Dynamic instrumentation
  • Snapshots support in programming languages and runtimes
  • Open source standardization around snapshots
  • The cost associated with observability
  • Google is applying to contribute Istio to the CNCF
  • Shopify case study for observability team

Resources:

Socials:


May 26, 202258:32
OpenSearch 2.0 and beyond with Eli - OpenObservability Talks E2E11
Apr 28, 202201:01:12
SLO Driven Engineering: from Dev to Prod - OpenObservability Talks S2E10

SLO Driven Engineering: from Dev to Prod - OpenObservability Talks S2E10

Google’s SRE Book popularized the concept of Service Level Objective (SLO) and the SLO-driven approach. But what does it really mean to make SLO driven decisions? How can we generate observability and synchronize teams around joint SLOs? And how can we automate SLOs and integrate them into the software release pipeline?

In this episode I’ll host Andreas Grabner. We’ll discuss the SRE practices, and how to automate SLO from dev all the way to prod. We’ll talk about the open source efforts to standardize the process under the Continuous Delivery Foundation, and about Keptn, the new CNCF open source project that promises to help with this automation.

Andreas Grabner (@grabnerandi) has 20+ years of experience as a software developer, tester and architect and is an advocate for high-performing cloud scale applications. He is a contributor and DevRel for the CNCF open source project keptn (www.keptn.sh). Andreas is also a regular contributor to the DevOps community, a frequent speaker at technology conferences and regularly publishes articles on blog.dynatrace.com or medium. In his spare time you can most likely find him on one of the salsa dancefloors of the world.

The episode was live-streamed on 15 March 2022 and the video is available at https://youtu.be/J81byOpVqrk 

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg


Show Notes:

  • What’s SRE
  • Where is SRE placed in the organization
  • SRE vs. DevOps
  • Good and bad SLOs
  • How to define SLOs top-down
  • Who owns SLO definition, monitoring, remediation
  • Where is SRE within less mature organizations
  • Keptn OSS project background
  • Who uses and contributes to Keptn project
  • What’s the CDF (Continuous Delivery Foundation)
  • Creating a standard CD event format under the CDF (CDF Events SIG)
  • Cloud Native Observability survey by the CNCF

Resources:

Socials:

Mar 30, 202259:56
Building web-scale observability at Slack, Pinterest & Twitter - OpenObservability Talks S2E09

Building web-scale observability at Slack, Pinterest & Twitter - OpenObservability Talks S2E09

What does it take to build observability in a web-scale company such as Slack, Pinterest and Twitter?

On this episode of OpenObsevability Talks I'll host Suman Karumuri to hear how he built these systems from the ground up on these #BigTech co's, about his recent research papers and more.

Suman Karumuri is a Sr. Staff Software Engineer and the tech lead for Observability at Slack. Suman Karumuri is an expert in distributed tracing and was a tech lead of Zipkin and a co-author of OpenTracing standard, a Linux Foundation project via the CNCF. Previously, Suman Karumuri has spent several years building and operating petabyte scale log search, distributed tracing and metrics systems at Pinterest, Twitter and Amazon.  In his spare time, he enjoys board games, hiking and playing with his kids.

The episode was live-streamed on 16 February 2022 and the video is available at https://youtu.be/IvidkV3TfYg 

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg


Show Notes:

* Who owns observability in large organizations?

* The gaps in current way of handling metrics 

* MACH research paper for metrics storage engine

* The gaps in current way of handling logs Slack KalDB

* SlackTrace - Slack in house tracing system 

Resources:

Socials:


Feb 27, 202258:39
SaaS Observability Done Right - OpenObservability Talks S2E08

SaaS Observability Done Right - OpenObservability Talks S2E08

SaaS (software as a service) is a popular model for many businesses today. SaaS businesses need agility to move fast and remain competitive. This means agility in the software IT stack, but also agility in the business models and product-led growth (PLG). Observability plays a key role in enabling SaaS organizations to move fast.

Achieving this agility, however, raises specific observability requirements. On this episode of OpenObservability Talks we’ll host Aviad Mizrachi, the CTO and Co-Founder of Frontegg, to help us map these requirements. Having escorted dozens of SaaS businesses across many verticals, Aviad brings a wealth of experience in how today’s SaaS is built and operated, and will share his insights and best practices on how to design and build the observability stack right.

Aviad has been a developer for the last 20 years. He held a few management and architecture positions on startups such as Vicon and HTS as well as in larger companies such as NICE and CheckPoint. Today at Frontegg Aviad works closely with many customers to help them build their SaaS solutions.

The episode was live-streamed on YouTube Live and Twitch on 11 Jan 2022 and the video is available at https://www.youtube.com/watch?v=ZcneTMeBPeg 

OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube. 

We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.
https://www.twitch.tv/openobservability
https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg


Show Notes:

  • What characteristics in today’s SaaS businesses dictate/influence the tech choices
  • How are SaaS systems built? Tech stack and architecture
  • Which observability is needed for SaaS?
  • Kubernetes & infra observability
  • Availability, responsiveness, low latency are critical in SaaS
  • product and business observability
  • Observability has many stakeholders
  • Recommended tooling for SaaS
  • Correlating different data signals
  • Persistence and the cost of storage
  • Final tips for SaaS observability
  • AWS recent outages and learnings
  • Log4j recent CVEs 

Resources:

  • AWS outages and learnings: https://horovits.medium.com/retrospect-on-the-aws-outage-and-resilient-cloud-based-architecture-cc513a32747

Socials:


Jan 31, 202258:18
Prometheus Pitfalls and the Rise of Continuous Profiling - OpenObservability Talks S2E07

Prometheus Pitfalls and the Rise of Continuous Profiling - OpenObservability Talks S2E07

We’ve grown to rely on “the three pillars” for observability - logs, metrics and traces. Popular frameworks such as Prometheus have helped popularize these practices. But now people are starting to realize that it’s not enough.

On this episode Dotan Horovits will host Frederic Branczyk for a discussion about the unspoken pitfalls of Prometheus and the challenges of current observability coverage. We will also discuss the rise of Continuous Profiling as a new observability signal, what it’s about and where it can help. We’ll also review the recent launch of Parca, an open source project for continuous profiling that traces its roots to Red Hat’s internal ConProf open source tool.

Frederic is the founder and CEO of Polar Signals. Before founding Polar Signals he was a senior principal engineer and the main architect for all things Observability at Red Hat, which he joined through the CoreOS acquisition. Frederic is a Prometheus and Thanos maintainer as well as the tech lead for the special interest group for instrumentation in Kubernetes. In a previous life, he was a security researcher working on key management solutions as well as intrusion detection systems. When not working on software Frederic enjoys obsessing over brewing a perfect cup of coffee.

The episode was live-streamed on 16 December 2021 and the video is available at https://www.youtube.com/watch?v=G02g63oI0IA 

OpenObservability Talks episodes are released monthly, on the last Thursday of each month. The episodes are also live-streamed on Twitch and YouTube Live - tune in to see us live, and pitch in with your comments and questions on the live chat.

Show Notes:

  • The limitations of the three pillars model of observability
  • Prometheus strengths and pitfalls
  • how to start with continuous profiling
  • how to correlate between different telemetry
  • Parca OSS intro
  • eBPF turned out perfect for instrumenting continuous profiling
  • Parca OSS future plan
  • how is the performance penalty of continuous profiling kept low
  • what's the solution for high cardinality in Prometheus?
  • will Parca OSS be contributed to an established OSS foundation?
  • Prometheus Agent mode released
  • OTEL operator now has an instrumentation CR
  • continuous profiling support for interpreted languages

Resources:

Social:


Dec 21, 202101:01:48
BPF origin story and the future of telemetry analytics OpenObservability Talks S2E06

BPF origin story and the future of telemetry analytics OpenObservability Talks S2E06

OpenObservability Talks S2E06: Hosting Steve McCanne  

We hear a lot about BPF in the industry today, applying this flexible technology to solve so many problems from routing, proxying, and of course observability. Correlating events and data from the operating system level across distributed systems is a key problem for the industry and community to solve. I am thrilled to announce Steve McCanne joining us for this episode. I have been lucky enough to spend time with Steve in my career and am delighted to have him join us to discuss the origin stories and where these foundational technologies might be applied in the future. Steve’s Bio and background speak for themselves.  

Steve McCanne is the "Coding CEO" at Brim, a small startup working on the open-source Zed Project and a new application called "Brim" that leverages Zed. Back in the days before the Web, Steve worked at the Lawrence Berkeley National Laboratory where he developed BPF, libpcap, the PCAP file format, and the tcpdump language and compiler, while also working on the Real-time Transport Protocol (RTP) for Internet video when the telcos claimed that real-time Internet communication was impossible without end-to-end virtual-circuit guarantees. (Guess who was right?) After a brief stint in academia in the late '90s, Steve crossed over to the dark side, became a tech entrepreneur, and never looked back. He has founded several startups and took his '02 company and Sharkfest's sponsor, Riverbed, public in '06.  

Resources  

The USENEX paper from 1993 on BPF architecture:  https://www.usenix.org/legacy/publications/library/proceedings/sd93/mccanne.pdf  

Open source tools Steve shared in the podcast:
https://github.com/brimdata/zed https://github.com/brimdata/brim  

Steve's GitHub BPF repo: https://github.com/brimdata/zbpf  
Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠


Nov 23, 202101:02:12
SRE at Google: Planet-scale observability - OpenObservability Talks S2E05

SRE at Google: Planet-scale observability - OpenObservability Talks S2E05

Have you ever wondered how services are operated at Google’s scale? Here’s your opportunity to find out. Ramón will share how his SRE team runs Google’s identity services, and the elaborate end-to-end observability they use to achieve it with strict SLA. We’ll also get a glimpse at the birthplace of Kubernetes, OpenCensus, Dapper, Monarch and other cornerstones of today’s cloud-native DevOps and observability.

Ramón Medrano Llamas (@rmedranollamas) is a staff site reliability engineer at Google, focused on user identity and authentication. He concentrates on the reliability aspects of new Google products and new features of existing products, ensuring that they meet the same high bar as every other Google service. Before joining Google in 2013, he worked at CERN developing and designing distributed systems for physics. He holds a master’s degree in computer science and is pursuing a PhD on distributed systems.

The episode was live-streamed on 26 October 2021 and the video is available at https://youtube.com/live/jVTZf1SXZrg


Show Notes:

  • scale and size of Google Identity services operation
  • evolution from monitoring to observability
  • telemetry collection
  • SRE job description is changing
  • Google Dapper
  • Google Census
  • operating end-to-end observability at scale
  • flexibility vs. runbook in SRE
  • how SRE at google different
  • transition from monolith to MSA
  • Linux Foundation launching a DevOps bootcamp
  • Parca OSS launched
  • how to intro SRE culture

Resources:

Oct 27, 202159:59
Observability Into Your Business And FinOps - OpenObservability Talks S2E04

Observability Into Your Business And FinOps - OpenObservability Talks S2E04

Observability is becoming a common practice for DevOps teams monitoring and troubleshooting IT systems. But Observability can offer much more than that. More advanced usage of telemetry, and in particular distributed tracing and its context propagation mechanism, can uncover insights into your business performance and can help solve business and FinOps problems.

On this episode of OpenObservability Talks I hosted Yuri Shkuro, creator of Jaeger project and a champion of Distributed Tracing, to discuss how tracing and observability can help beyond DevOps, whether on business cases, FinOps or even software development. We also caught up on the latest updates from Jaeger, the CNCF’s distributed tracing OSS project, its synergy with OpenTelemetry and more topics.

Yuri is a software engineer who works on distributed tracing, observability, reliability, and performance problems; author of the book "Mastering Distributed Tracing"; creator of Jaeger, an open source distributed tracing platform and a graduated CNCF project; co-founder of the OpenTracing and OpenTelemetry CNCF projects; member of the W3C Distributed Tracing Working Group.

The episode was live-streamed on 14 September 2021 and the video is available at https://youtube.com/live/YSOyTagKGtM

Show Notes:

  • why distributed tracing?
  • tracing through async flows
  • why is the slow adoption of tracing?
  • instrumentation challenge for tracing adoption
  • using context propagation for business use cases
  • observability tooling maturity
  • Jaeger project updates
  • OpenTelemetry accepted to CNCF incubation
  • Cortex and Thanos accepted to CNCF incubation
  • Google contributing SQLCommenter project to OTel
  • K8s v1.22 releases API Server tracing in alpha

Resource:

Sep 19, 202101:06:47
Fluentd for logging and metrics and path forward - OpenObservability Talks S2E03

Fluentd for logging and metrics and path forward - OpenObservability Talks S2E03

In this episode, we’ll talk with industry veteran and product manager Anurag Gupta who has been working in open source observability for over 4 years. We will go into depth on his background, and how he views the ecosystem of open source. Then we will dig into the Fluentd and Fluent Bit projects and discuss some of the amazing innovations coming from this project. Learn what’s next for logging, and how a consolidated data collection plane is being driven by the Fluentd project.

Aug 26, 202146:22
Prometheus, OpenMetrics, and the CNCF Observability Ecosystem - OpenObservability Talks S2E02

Prometheus, OpenMetrics, and the CNCF Observability Ecosystem - OpenObservability Talks S2E02

The CNCF has a rich suite to address monitoring Kubernetes and cloud-native workloads. First of which is Prometheus, which is widely adopted, with great out-of-the-box compatibility with Kubernetes. But under the CNCF you can also find OpenMetrics that offers standardization of the metrics format, Thanos and Cortex which offer long-term storage for Prometheus, and other complimentary solutions and integrations.   

On this episode of OpenObservability Talks we’ll host “RichiH” Hartmann and discuss the different OSS projects, the synergy between them, and the future roadmap in building the community and making CNCF a leading offering.  

Richard "RichiH" Hartmann is Director of Community at Grafana Labs, Prometheus team member, OpenMetrics founder, CNCF SIG Observability chair, and other things. He also organizes various conferences, including FOSDEM, DENOG, DebConf, and Chaos Communication Congress. In the past, he made mainframe databases work, ISP backbones run, and built a datacenter from scratch.

The episode was live-streamed on 02 July 2021 and the video is available at https://youtube.com/live/j3nFFHSosnI


Show Notes:

  • OpenTelemetry accepted to CNCF incubation
  • OpenTelemetry structure
  • OpenTelemetry community adoption
  • OpenMetrics and Open* confusion
  • OpenMetrics and OpenTelemetry synergy
  • OpenMetrics updates
  • CNCF’s Observability TAG (Technical Advisory Group)
  • How to sync between projects on CNCF
  • Prometheus state and roadmap
  • Prometheus conformance program
  • Thanos and Cortex projects
  • how the tech stack benefits humans
  • Grafana, Loki and Tempo projects

Resources:

Jul 25, 202101:09:58
Codeless Kubernetes Observability with eBPF - OpenObservability Talks S2E01

Codeless Kubernetes Observability with eBPF - OpenObservability Talks S2E01

Current observability practice is largely based on manual instrumentation, which creates a barrier to entry for many wishing to implement observability in their environment. This is especially true in Kubernetes environments and microservices architecture.

eBPF (extended Berkeley Packet Filter) is an exciting new technology for Linux kernel level instrumentation, which bears the promise of no-code instrumentation and easier observability into Kubernetes environments (alongside other benefits for networking and security).

On this episode of OpenObservability Talks we’ll host Natalie Serrino, Principal Engineer at Pixie Labs, which was recently acquired by New Relic. We’ll talk about observability in Kubernetes environments, eBPF and its use cases for observability.

We’ll also talk about Pixie, the Kubernetes-native in-cluster observability platform, and the exciting news of it being open sourced and contributed these days to CNCF under Apache 2.0 license.

Natalie is a Principal Engineer and Tech Lead at New Relic. She works on the Pixie auto-telemetry observability platform, which was acquired and open sourced by New Relic. She focuses primarily on Pixie’s data layer, including its query language, compiler, and query execution engine.

The episode was live-streamed on 20 June 2021 and the video is available at https://youtube.com/live/NYDBj5ctKaw


Show Notes:

  • challenges in k8s observability
  • state of instrumentation
  • automatic instrumentation
  • eBPF overview
  • eBPF vs. service mesh side cars
  • Pixie project overview
  • Pixie’s roadmap and integration plans with CNCF ecosystem
  • Netflix engineering sharing use case of eBPF
  • instrumenting with Istio
  • opensearch RC1 released
  • K8s unpredictable spend
  • logs aren't enough, need tracing - recommended article

Resources:

http://www.brendangregg.com/ebpf.html

https://blog.px.dev/

https://docs.px.dev/about-pixie/roadmap/

https://www.businesswire.com/news/home/20210504005480/en/New-Relic-Joins-Cloud-Native-Computing-Foundation-Governing-Board-and-is-in-the-Process-of-Contributing-Pixie-Open-Source-for-Kubernetes-Native-Observability

https://netflixtechblog.com/how-netflix-uses-ebpf-flow-logs-at-scale-for-network-insight-e3ea997dca96

https://logz.io/blog/istio-instrumenting-microservices-distributed-tracing/

https://opensearch.org/blog/update/2021/06/opensearch-release-candidate-announcement/ 

https://thenewstack.io/tracing-why-logs-arent-enough-to-debug-your-microservices/ 

https://www.theregister.com/2021/06/29/kubernetes_spend_report/


Socials:

Twitter:⁠ https://twitter.com/OpenObserv⁠

YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠



Jun 30, 202156:51
OpenSearch: The Open Source Successor of Elasticsearch? - OpenObservability Talks S1E12
May 27, 202101:01:19
Diving deep into Jaeger and OpenTelemetry with Juraci Paixão Kröhling - OpenObservability Talks S1E11

Diving deep into Jaeger and OpenTelemetry with Juraci Paixão Kröhling - OpenObservability Talks S1E11

We are thrilled to have Juraci Kröhling a Software Engineer at Red Hat; CNCF, Maintainer for Jaeger, and OpenTelemetry. He will be live and in-person this month on the podcast in a discussion with Jonah Kowall who is the CTO at logz.io and contributor to Jaeger, OpenTelemetry, and OpenSearch.

Apr 30, 202156:29
Interoperability of open-source observability and new signal in the neighborhood, profiling! - OpenObservability Talks S1E10

Interoperability of open-source observability and new signal in the neighborhood, profiling! - OpenObservability Talks S1E10

Join Jonah Kowall and Bartek Plotka for a discussion on the latest happening topics on open source observability. Bartek works on many projects in open source and is Principal Software Engineer at Red Hat; CNCF SIG Observability Tech Lead. He is very active in the community as one of the leaders of Prometheus, Thanos, OpenMetrics, and many other projects.

Mar 29, 202158:47
How Much Observability Is Enough? - OpenObservability Talks S1E9

How Much Observability Is Enough? - OpenObservability Talks S1E9

The ninth of our OpenObservability Talks has Jujhar Singh, Global DevSecOps Practice Lead at The Economist.

How much observability is enough? What is the investment required to achieve it? How can we drive observability in the company in a measured and pragmatic way? 

This was first live-streamed on 25 February 2021 and the video is available at https://youtu.be/sR-Q3Z-YP2E


Show Notes:

  • How to drive observability in your organization
  • What is the minimum observability needed for your organization?
  • The tech stack impact on observability needs
  • Direct correlation between organization, product and observability
  • How to assess your observability needs
  • The investment involved in observability
  • eBPF and tools for deep Linux inspection
  • OpenSearch (Elasticsearch fork) status update
  • OpenTelemetry’s Tracing specification reaches v1.0
  • Stanza contributed its logging agent to OpenTelemetry
  • Docker was contributed to CNCF

Resources:

Socials:


Feb 28, 202159:49
Put the OPEN in Observability: Elasticsearch and Kibana relicensing and community chat - OpenObservability Talks S1E8

Put the OPEN in Observability: Elasticsearch and Kibana relicensing and community chat - OpenObservability Talks S1E8

The eighth of our OpenObservability Talks has Tomer Levy, CEO & Founder of Logz.io.

The community is in turmoil around Elastic's announced plan to take Elasticsearch and Kibana off open source. In this episode, both Dotan and Mike have the pleasure of hosting Tomer where we discuss the recent news of Elastic moving Elasticsearch and Kibana to a dual non-OSS license - SSPL and Elastic License - and the implications that have on the open source community around it, including plans to fork Elasticsearch and Kibana, AWS announcement and more. We also talk about what Logz.io hopes to do, and how it wants the OSS to be better than ever.

Tomer Levy is co-founder and CEO of Logz.io. Before founding Logz.io, Tomer was the co-founder and CTO of Intigua, and prior to that he managed the Intrusion Prevention System at CheckPoint. Tomer has an M.B.A. from Tel Aviv University and a B.S. in computer science and is an enthusiastic kitesurfer.

The live streaming of the OpenObservability Talks is on the last Thursday of each month, and you can join us on Twitch or YouTube Live.

Socials:

Jan 28, 202137:57
An Observability chat with Andy Thurai - OpenObservability Talks S1E7

An Observability chat with Andy Thurai - OpenObservability Talks S1E7

The seventh of our OpenObservability Talks has Andy Thurai, Senior Analyst at GigaOM.

Jonah Kowall, CTO and Andy Thurai will talk about Andy's career and journey as a vendor and an analyst. We will discuss the observability market along with APM and other aspects of monitoring. We will then dive into the open-source ecosystem and how this is changing vendor thinking. Of course, we'll also be discussing OpenTelemetry!


Andy Thurai is the Founder & Principal at the FieldCTO providing content and advisory services to enterprise customers in particular on AIOps, CloudOps, AI, ML, and Observability areas.


He is an accomplished IT executive, strategist, advisor and evangelist with 25+ years of experience in executive, technical and architectural leadership positions at companies such as IBM, Intel, BMC, Nortel and Oracle; he advises many start-ups, and he is a Steering Committee Member for AIOps Exchange. He has been a keynote speaker in many major conferences, as well as a host of many webcasts, podcasts and video chats. He is a regular Forbes contributor and has written 100+ articles on emerging technology topics for publications such as Forbes, AI World, VentureBeat and Wired. Andy Thurai can be reached on Twitter at @AndyThurai, or on LinkedIn.

This was first streamed at https://www.twitch.tv/openobservability on December 29th and the full video is available at https://youtu.be/Hr4lGqLiMa0

The live streaming of the OpenObservability Talks is on the last Thursday of each month, and you can join us on Twitch or YouTube Live.

Socials:

Dec 31, 202044:21
All Metrics Are Wrong, Some Are Useful - OpenObservability Talks S1E6
Nov 29, 202001:13:05
Microservices Observability for Developers and Pre-Production - OpenObservability Talks S1E5

Microservices Observability for Developers and Pre-Production - OpenObservability Talks S1E5

The fifth of our OpenObservability Talks has Michael Haberman, CTO & Co-Founder of Aspecto.

There's a lot of discussion on how to achieve observability in microservices. Most of the discussion revolves around production workflows such as granular monitoring or debugging of complex environments.

A much less discussed aspect is the development stage, before even reaching production. In this talk we will discuss the challenges of microservices environments, and how to leverage tracing for pre-production purposes such as improving your tests, understand dependencies and more.

Michael Haberman is a veteran in the microservices space and brings significant experience as a software architect and a consultant. Michael  shared his experience on achieving observability in microservices environments in general, and during development stages in particular, with some useful tips on how to solve production issues, before they reach production.

This was first streamed at https://www.twitch.tv/openobservability on October 29th and the full video is available at https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

The live recording of the OpenObservability talks is the last Thursday of each month, and you can join us on Twitch or YouTube Live.

Socials:

Twitter: https://twitter.com/OpenObserv

Twitch: https://www.twitch.tv/openobservability

YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg

Website: https://openobservability.io/

Links Shared in the recording:

OpenTelemetry's Tracing Specification Reaches Release Candidate

A new open source project: Promscale, An analytical platform and long-term store for Prometheus, with the combined power of SQL and PromQL

AWS announcing Public Preview of AWS Distro for OpenTelemetry

Oct 31, 202058:12
Distributed Tracing at Uber-Scale - OpenObservability TalksS1E4

Distributed Tracing at Uber-Scale - OpenObservability TalksS1E4

The fourth of our OpenObservability Talks has Albert Teoh from the Jaeger project.

Distributed tracing has been gaining momentum with the growing popularity of microservices. Jaeger is a popular open source tool originally developed at Uber and now part of the CNCF (Cloud Native Computing Foundation).

Albert had been working at Uber for the past 2.5 years, where he got hands-on with Jaeger. On this talk Albert will share with us his experience with distributed tracing, from introducing it into new code all the way to production. Albert will explain important concepts and considerations, and will discuss common challenges and solutions in introducing distributed tracing in an existing large scale system.

This was first streamed on 24 September 2020 and the full video is available at https://youtu.be/35aInRLbTQo

The live recording of the OpenObservability talks is the last Thursday of each month, and you can join us on Twitch or YouTube Live. 


Socials:

Twitter: https://twitter.com/OpenObserv 

YouTube: https://www.youtube.com/channel/UCLKOtaBdQAJVRJqhJDuOlPg


Resources:

why Jaeger was built by Uber Engineering: https://eng.uber.com/distributed-tracing/

Domain oriented microservice architecture: https://eng.uber.com/microservice-architecture/

Reference to resource usage attribution, from the book "Mastering Distributed Tracing" by Yuri Shkuro: https://subscription.packtpub.com/book/networking_and_servers/9781788628464/2/ch02lvl1sec28/resource-usage-attribution

CNCF published the second quarterly CNCF End User Technology Radars; the topic for this Technology Radar is observability https://www.cncf.io/blog/2020/09/11/cncf-end-user-technology-radar-observability-september-2020/

Crtex & Thanos voted into CNCF incubation https://www.cncf.io/blog/2020/08/20/toc-welcomes-cortex-as-an-incubating-project/  https://www.cncf.io/blog/2020/08/19/toc-approves-thanos-from-sandbox-to-incubation/

Kubernetes v1.19 release offers structured logs for the system components https://logz.io/blog/a-practical-guide-to-kubernetes-logging/

Sep 30, 202058:50
Long-Term Metrics with M3 and Prometheus - OpenObservability Talks S1E3

Long-Term Metrics with M3 and Prometheus - OpenObservability Talks S1E3

Episode 3 of OpenObservability Talks, where we'll be hosting Matt Schallert of  Chronosphere. The topic for this episode is Long-Term Metrics with M3 and Prometheus.

Prometheus is the de facto solution when getting started with metrics, supporting use cases where a single node can store the entire dataset. When it comes to scaling out workloads or storing long-term data, Prometheus offers the Remote Write and Read interfaces to integrate with other storage solutions. 

In this talk, Matt will show attendees how they can scale their existing Prometheus metrics workloads with M3, an open-source metrics platform originated at Uber. Matt will discuss the background of Prometheus’s remote storage interfaces, and how they enable users to integrate with any number of systems they’re familiar with. He will demonstrate how quickly users can get an M3 stack up and running and integrate it with Prometheus.

The audience will walk away with a better understanding of the challenges of effectively scaling their metrics workloads, and how they can overcome these challenges with open source tools such as M3.

This was first streamed at https://www.twitch.tv/openobservability on August 27th and the full video is available at https://www.youtube.com/watch?v=XQ_yCLyAMLU

Links Shared in the recording:

If you have a talk or a subject you'd like to talk about send it here https://forms.gle/pTVwDMCP1fK32tAF9

Socials:

Aug 27, 202053:39
A chat with Paul Bruce and Jonah Kowall about the OpenTelemetry community - OpenObservability Talks S1E2
Jul 30, 202001:11:29
Five Tips on Application Monitoring with Prometheus - OpenObservability Talks S1E1

Five Tips on Application Monitoring with Prometheus - OpenObservability Talks S1E1

Welcome to the first-ever OpenObservability Talks!

This was first streamed at https://www.twitch.tv/openobservability on June 25th and the full video is available at https://www.youtube.com/watch?v=Otb57MNZfKA

You'll get an introduction to the series, a talk by Jack Neely of 42 Lines on Five Tips on Application Monitoring with Prometheus, and a run-through of current events in the world of Open Source Observability and the things that affect it.

If you'd like to submit a session for a later event then just drop it in the CFP here https://forms.gle/Le691dn7CU35GwKq7

Socials:

Jun 29, 202039:33