theCUBE with Observe CEO Jeremy Burton: Developer Experience, Security, and GPT in Observability
By Observe Team May 5, 2023
The way we built Observe was quite, what I call un-opinionated, in that we can accept observability data from Kubernetes clusters, AWS, or from whatever application, but we can also accept security log events.
Introduction
While at RSA, Observe’s CEO, Jeremy Burton, sat down with John Furrier and Dave Vellante of theCUBE to talk about observability market changes, the role of emerging tech like AI, and Observe’s latest App, Threat Intelligence. You can check out the entire conversation in the video below, or read some excerpts on key topics.
On The Importance of Developer Experience
John: What we’re finding in open-source at KubeCon and the CNCF, the developers are the new consumers. If the developers like it it’s consumed and becomes adopted, and that has nothing to do with marketing. So if the developers like a product and it helps their job, it’s easier, and it works then all the VCs are looking at that as a metric on the investment side and on the traction side. So combining that ease of use and doing all the heavy lifting on the code this is the new reality. Can you unpack that on the software side and expand on this developer adoption dynamic?
Jeremy: Yeah, it used to be the case, back when I started my career at Oracle in the mid-90s. We had a great sales team and at times, there were certain versions of Oracle that were not that great. But a great sales team could sell it to senior people and then sort of force it on the technical folks in the organization. You can’t do that anymore, right?
I think certainly you’ve still got to sell to the people who’ve got money in the budgets, but you can’t jam it down the throats of software engineers, because you’re going to get tissue rejection. So even enterprise software now it’s got to be much slicker than it ever was in the past. There’s got to be less friction. It’s got to be geared toward a great developer experience. If you do that, people will pick it up, and start using it.
On Building for Observability
Dave: So what’s the state of play today? Because you have companies that were doing metrics or whatever, legacy companies that now say, “Hey, we’re observability too.” Guys like yourself you had a less mature stack, so you’ve got the modern play, and then it’s this big mishmash of folks saying “We’re observability too.” So how should we think about that?
Jeremy: So when we set out to build Observe, we didn’t think about just logs or just tracing or just metrics. We thought about, well, how do we build a system that can ingest any kind of unstructured event, whether it’s a log event, a trace event, a time series metric or events come out of salesforce.com or out of ServiceNow or wherever those events may be, how can we just suck them into one big Snowflake database?
That’s another change because Snowflake didn’t really exist when folks like Datadog and Splunk and all these folks were around, they couldn’t build on top of Snowflake because it didn’t exist. So we took advantage of new disruptive technology, Snowflake, and we thought, why can’t we just build one big event database, stream everything in there, then we can start to ask questions that go across these silos. And that’s really where the value in the product and observability is.
I see an alert over here that tells me a metric has been exceeded. Okay, where are the logs for that user session? Now show me the traces. And you don’t actually have to leave the system in order to see all the relevant context you need to figure out root cause.
On Security and Observability
Dave: So it’s kind of an adjacency to security. I’d like you to help us understand where it fits. I don’t think you’re repositioning as a security company, but you’re an adjacency, are you not?
Jeremy: Yeah, it’s getting hard to tell. I mean, there are certain companies in our space that set out to build an observability tool. So like LightStep that ServiceNow acquired or Honeycomb, they’re probably never going to do security. Why? Because they built an observability tool.
The way we built Observe was quite, what I call un-opinionated, in that we can accept observability data from Kubernetes clusters, AWS, or from whatever application, but we can also accept security log events. So things coming out of Palo Alto Networks’ firewalls or identity management systems, it’s just another event. So the smaller companies that we started selling to when we first had a product, it turns out the DevOps team is actually the SecOps team, right? It’s usually one or two people. And they’re like, “Oh yeah, security, we do that too. Hey, by the way, can we ingest this stuff and can we pull out IP addresses?” And we say, well, yeah, you can do that.
And I’d say about 20% of our customers are now doing this kind of thing so much so that we were like, okay, maybe we should productize this. So we actually have a threat intelligence product at Observe. It’s brand new. I don’t even think we’ve announced it yet, but there you go.
We hired the guy that did Splunk Enterprise security; we thought he might know a thing or two and he’s produced this threat intelligence module so folks can take sort of blacklisted IP addresses from external open source feeds and correlate what they see externally with IP addresses that are sitting in their log files internally. I sort of joke that we’re building a SIEM for people who don’t want a SIEM.
Embracing New Tech and AI
John:
You were on the right side of this trend. And I would say that you guys have pretty much made good calls. Where is the market relative to the people who aren’t making the right calls? You and I both talked a lot, I think two times ago you were on theCUBE, about how overfunded the sector was with observability.
How do you see the sides of the street and the ones who cross over to the right side of the history of this distributed computing, cloud-native, new modern app-centric, open-AI… There’s a whole nother tsunami coming. Who makes it and who doesn’t?
Jeremy:
In tech we love fashion, we love a new trend. In the observability space, I was not a fan of AI because I didn’t think it worked. I thought it was sort of snake oil because these apps, modern apps, at least, they’re changing every day. So you can’t build a model that’s worth anything because you’re going to get lots of outliers. So it didn’t really have a lot of good applicability for modern distributed apps that change quite often.
I think GPT comes along and you think, oh, wait a minute here. It’s not at the stage where you can just throw terabytes of machine data at an LLM, and it’ll tell you the root cause of a problem, but you can feed it lots of great technical information. We feed it code examples from our language, Opal, and it can write code for you.
It can help you get up to speed on a new product very quickly. If you see an error condition, it might not tell you root cause but it can tell you the series of steps that you need to take in order to figure out what the root cause is. So I think this could improve the productivity of this kind of tooling by 20% or 30%.