Skip to main content

Search complete. 57 mentions across 32 episodes found for "UTF-8".

Nov 20, 2026

Stuart CheshireGUEST
19:32
And given that you don't type these things on the command line, we're free of that restriction.
Stuart CheshireGUEST
19:36
So any UTF-8, Kanji, whatever characters you want are fully supported, and they go in the DNS packet as UTF-8.
Stuart CheshireGUEST
19:48
AppleTalk used to have a structured namespace with name, type, and zone.
Stuart CheshireGUEST
19:53
And we have a similar thing here.
Lane WagnerHOST
12:07
You mostly don't have to think about it after that.
Lane WagnerHOST
12:09
Like under the hood, you know, once you've learned a little bit more about how computers work, you understand that like strings are effectively like UTF-8, typically UTF-8 encoded arrays that are, you know, have to be dynamically allocated because you can make them very large or very small.
Lane WagnerHOST
12:25
And you have to deal with that and see Whereas in a language like Python or even Go, which is much, I shouldn't say much, but like, you know, Go is a much more efficient language at least than Python.
Lane WagnerHOST
12:35
And it gets you a little closer to the hardware than a language like Python does.
David ShapiroHOST
28:15
It was 42 chunks total, and I predicted 42.75, so spot on.
David ShapiroHOST
28:24
When it's encoded as UTF-8, it's about 1 to 1 in terms of characters, and 1,000 characters is roughly 1 kilobyte, put it that way.
David ShapiroHOST
28:35
Okay, so all that being said, here's the output.
David ShapiroHOST
28:41
So you see where we've got the double new line.
David ShapiroHOST
39:17
Um, Prompt encoding, I don't think we're gonna need that because we're not using anything from...
David ShapiroHOST
39:28
So this line of code, I had it here because there's something funky in the files from Gutenberg where I needed to encode it to ASCII, ignore errors, and then decode it back to a string because it used some form of UTF-8 that GPT-3 was not happy with.
David ShapiroHOST
39:46
It kept erroring out.
David ShapiroHOST
39:48
But I don't think we need that.
Percy LiangHOST
78:01
So you can turn strings into bytes.
Percy LiangHOST
78:04
So Unicode has a UTF-8 encoding which means that you can sometimes a string like A is just one byte and sometimes a string is multiple bytes.
Percy LiangHOST
78:19
So let's build a tokenizer around that.
Percy LiangHOST
78:22
So we can take this string and convert it into a sequence of bytes.
David ShapiroHOST
29:41
Okay, so the first thing that I did was I tried to go and change the encoding of the books.
David ShapiroHOST
29:47
So encoding, it was UTF-8 bomb.
David ShapiroHOST
29:51
So I just converted it to UTF-8.
David ShapiroHOST
29:53
That didn't fix it.
David ShapiroHOST
29:54
So I was like, okay, GPT-3 should be able to accept all UTF-8, but there's some artifact in this that it didn't like.
David ShapiroHOST
30:03
So here's what I did was I said, okay, let's open the prompt and we'll encode it to ASCII, which is much simpler than UTF-8, and then we'll decode it back to a normal string.
David ShapiroHOST
30:17
So it's basically saying, okay, whatever this is, ignore the errors, simplify this codec, make it ASCII standard, and then decode it, and then GPT-3 likes it now.
David ShapiroHOST
30:29
So it's recording this.
Haiper SafiGUEST
37:37
Let's say you wanted the AI to write in Danish.
Haiper SafiGUEST
37:39
The UTF-8 was not great.
Haiper SafiGUEST
37:41
So you have funny words in there.
Haiper SafiGUEST
37:43
Sometimes it would just go on its own path and trying to figure out, OK, how am I supposed to spell this in Danish? When I started this from the very beginning, I mean, when AI was out, I really started to try to see what it could be used for.
Chetan RaghuprasadGUEST
18:57
So this is direct.
Chetan RaghuprasadGUEST
18:58
I mean, direct and also another advantage advantage of this is uh it has the capability to like simply validate an arbitrary string of data which is passed to it uh in the form of like utf-8 text or base64 encoded binary data and then it directly records it to the transactions verified execution log So that's the difference and it is quite straightforward to the attackers.
Chetan RaghuprasadGUEST
19:30
And also the Solana transactions fees are like practically microscopic.
Chetan RaghuprasadGUEST
19:38
So it is very minimum and the attackers can rotate their dead drops or the configuration data that they have written into like almost virtually at zero cost.
Tech Rants

Tech Rants

Lean Pi Mail Hub

Nov 18 · 1 Mention

Edward OjamboHOST
46:04
This is really impressive.
Edward OjamboHOST
46:07
And character encoding, I know it's UTF-8 because I'm on Linux.
Edward OjamboHOST
46:11
This one here, Unicode, that's the, although it says automatic, that's the one, but you can change it to whatever you want.
Edward OjamboHOST
46:18
I don't know if Windows is still using the, one of these ones, the Western European ones, I'm not sure, the, what is it, UTF-16.
David ShapiroHOST
30:22
So this is just a brief function that I have that just I recycle it to open files.
David ShapiroHOST
30:29
It defaults to UTF-8 and then it just returns the file object.
David ShapiroHOST
30:34
Then I have another function to save the conversation.
David ShapiroHOST
30:37
It saves it to the convos directory and it gives you the topic, but also puts in the time.

22 more episodes mention UTF-8.

Create an account to see the whole feed, search across every transcript, and follow the entities you care about.

We value your privacy

We use cookies to understand how you use our platform and to improve your experience. Click “Accept All” to consent, or “Decline non-essential” to opt out of non-essential cookies. Read our Privacy Policy.