A new Go API for JSON
https://go.dev/blog/jsonv2Or more concretely, if we had a parser generator + json parser specification that can generate efficient simdjson style parsers, then we should be able to parse json restricted to a subset of json based on a schema at least as efficiently as simdjson can parse general purpose json.
That way we could have the best of both worlds, a (potentially lazy) statically type-safe model and simdjson performance!
I still think that this is definitely doable, it's just that a lot of infrastructure needs to be built for it.
I remember reading that Meta is doing something similar with binary formats. I find it weird that there's nothing like that for json yet.
I think most people who use JSON don't want strong typing or to use schemas. If you do, you just use a different serialization format: JSON is designed to be a generic, human-readable way to basically represent any kind of data structure (since it deserializes to JS objects). So everything about JSON is working against you if you do try to enforce type/schemas on it.
A grpc-like protocol with more prescriptive type registries/discovery (over the network) and better interop with http (like grpc-gateway) would be the best approach IMO. Definitely not JSON schema
Not necessarily. It's biggest advantage is probably that it has very good interoperability. Basically every language has a readily accessible library for json, and even if you don't, it isn't that hard to write a parser and/or generator for it. And being human readable isn't something to sneeze at either.
Clearly it isn't, otherwise every language would have sorted it out by... a long time ago.
You can still use it though, and I built my tiny decimal library around it: https://github.com/ncruces/decimal
A JSON parser MAY accept non-JSON forms or extensions.
An implementation may set limits on the size of texts that it accepts. An implementation may set limits on the maximum depth of nesting. An implementation may set limits on the range and precision of numbers. An implementation may set limits on the length and character contents of strings.
Add to that things like duplicate keys being allowed by the spec. There’s a lot of implementation choices to decide on.
And then things can get really complex with JSON Schema.
I wish they'd left `MarshalIndent` in there though. The equivalent code is super complicated and non-obvious, so it's a pain to migrate, and it's a pain when you're banging out some quick code and want to dump some readable JSON. I suppose these days the LLMs can write the complicated version for us, of course.
v := jsontext.Value(b)
v.Indent()
fmt.Println(string(v)) func WithIndent(indent string) Options
json.Marshal(v, jsontext.WithIndent(" "))
?There is no introspection and speed is superb. I would definitely recommend jsontext as a parser for anything non-straightforward.
I'm just getting the neuron-band back together where Golang is concerned. I have done some in the past for little system tasks, but I am switching away from interpreted languages entirely for web stuff, and initially to Go, as I switch away from hand-coding to at least LLM-supported coding.
So I am a bit out of date on the pain points (I never really felt pain points when I last used it, because the jobs I was doing had very small, quite Go-friendly shapes).
I'm not entirely surprised, and I'm glad the Go team is carefully considering their interfaces before launching. But 6 years seems like a lot of work to land a JSON API.
JSON processing, quite markedly, is renown in Go for being slow, so I don't really understand why it took so long
Within our company I saw people in other teams writing their own optimised JSON libraries for whatever hellish reason. I always reiterated the point that as soon as that app made a single IO request all those nanosecond savings are invisible. Yes, it affects your total CPU time for processing a request but compute was never ever a limitation for us and it's likely not a limitation for most people. And if the trade off is your app needs scale slightly larger versus writing custom JSON code or pulling in a third party dependency, then I'm accepting the larger scaling.
The biggest benefit of encoding/json/v2 isn't the speed, it's the improved API.
I kind of like that the standard library doesn’t move fast. It makes it easier to fit the ecosystem into my head (unlike, for example, the JavaScript ecosystem where I find code doesn’t survive 2 years before failing to compile).
But I do agree that slow and steady produces its own problems. Your example of generics is a good one. But think the example of Go modules is another, perhaps even better, example of how sometimes the Go team really should have accelerated their efforts on.
Only about a year from when the Haskell guys lent their expertise in designing it and it landing in a release. That's actually fairly quick given that it was a pretty significant change under the hood and constrained by only two releases annually.
The Go team was always clear that they didn't have the necessary capabilities internally, despite a lot of failed efforts trying anyway, and were relying on the open source community to lend a hand. Unfortunately, open source doesn't mean open willingness to contribute. It turns out that for all the experts on HN, posting "go doesn't have generics lol" was preferable to helping out.
So yea hopefully a nice new addition but only my prompts now matter. So need to make sure they pickup on the new API.
I am using local LLMs and Go has a bunch of strengths where smaller contexts are concerned; less complicated flow of control being one of them.
I hope you're not still doing this, it's nothing to brag about.
If you code in Go, the code is more high level, prettier, simpler.
If you code in Rust, the code is more robust, efficient, faster.
Both are important for LLM, why can't we have a high level, simple language that is also robust, efficient and fast? =)
I think many variants from Rust can be stripped to make it simple, especially in type system, which is way more complicated than go.
Go is about as far as you can "go" to the efficiency end without becoming Rust. It's a low-overhead language (like C) but with a GC that ensures memory safety. If you want to manage memory yourself and still prove safety it gets very complicated, as we see in Rust.
The few times I've needed a language without a GC and used Rust and C as target languages, I've reviewed the code less and enjoyed it less.
First, I like programming and don't want to stop doing it just because a machine can do it too. Second, I'm pretty sure atrophied coding skills will make me worse at directing the machines, thus worse at what seems to be "the job" these days.
In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding? I mean isn’t whats left just design and architecture and testing?
Sure, if LLMs are producing shit code you need to have an understanding of what it should do instead. But I think that day is already gone.
Its hard to think of a skill that you’d need to build with an LLM that requires writing meaningful amounts of code by oneself. From my end all the complexity is business logic and system design.
Agents make it too easy to gloss over too much. They're great at work where i care about productivity but I've zero interest at home where the point is learning and having fun
Why? Because I don’t give time to wait for code to compile on my time and I don’t have time for people to learn a new language for review at work.
Life is good! Would recommend that split.
However it kind of makes me feel like my professional life was wasted and I don't really know what kind of future I have. One of Go's main points was you can pick it up in a weekend. What career moat do people like us have?
I'm happy it's getting fixed!
Completely unrelated: can anyone tell me why language authors introduce in/out/inout function parameters (see also C#).
It feels like this is some rather bad workaround for something that breaks assumptions about parameters being passed into the function.
If the allocation is owned by the calling function which uses it internally and then doesn’t return it, it can be a stack variable. Or the same allocation can be reused many times in a loop. Or an array of allocations can be made all at once with foreknowledge of the total number.
But you might be conflating that with in/out for Contra/Co-variance when specifying a generic.
They are one way to implement multiple return values. For multiple returns, tuples can be used. In fact, async methods can’t declare out params
I feel that is the kind of broken default that will come back and bite a bunch of people, and will eventually need to be fixed in a v3.
type ListUsersResponse struct {
Users []User `json:"users"`
}
userRows := db.Query(...)
var users []api.User
for _, user := range userRows {
users = append(users, api.User{...})
}
This is definitely not what I expect in other language. I would argue this is a quick of Go though and not JSON's fault.
Not necessarily. Among other things, MarshalJSON methods called during marshaling can return an error for whatever reason the developer sees fit - one possibility that comes to mind is that part of the object tree being marshalled is lazily loaded, and failed to do so.
And, as others have mentioned, there are types like "any" whose marshalability can't be determined at compile time. Those types aren't necessarily at the top level, either; they could appear anywhere in the object tree.
Because they are common types, makes sense to precautionary have the error return.
"What if it sucks?"
"There are now TWO ways to do each thing! No more than TWO! I promise!"
It's like python again.