PageRank

So the ontology is that we need to compute that index offline all the time.

Um, so it's almost like the PageRank algorithm that Google had invented back in the day, but it's more complicated.

'Cause Google was just looking at a web where everybody can go on the web.

Here, there are permissions.

So I think there's that perspective, is the long-tail distribution means we can create relevant content, um, that targets there.

And then I think there's the authority, uh, consideration is, okay, well, if everyone has relevant content, how does the model then reason over who to trust? Who does it decide who to trust? Um, historically, with traditional SEO, you know, we look at PageRank.

Um, it's, you know, um, how we know Google assesses authority of pages and domains.

Again, it's, it comes down to consensus.

Because I have this big monstrous graph, I wanna keep finding out what are those sets of related people, right? What are these communities? It's, what are the clustering things that you're trying to do? And once you detect these suspicious clusters, some of them may be suspicious, some of them may be not, you can now, you want to zero in on who are the most influential people within that suspected cluster.

Now, you can squint at this problem and say, "What is this like?" Actually, it's just like PageRank.

In fact, it is PageRank.

You can run PageRank on the nodes within that cluster.

And so although PageRank was originally for web pages, it's actually really very, very good at identifying who are the most influential nodes or the central nodes in the network.

So not only can you figure out here is the colluding set, you can now zero in on your actual problem space.

So I think this is a, kind of a great, great example of how you could actually run these things, and it's really valuable to be able to do this in situ without having to move this to some other system.

They can potentially flag or block transactions, and this way the real time comes in.

I want all AIs, I want Grok to train on it because the alternative is that it's going to, you know, train on Mecca Hitler stuff, right?

So do these LLMs have a concept like Google's PageRank where they assign some sort of quantitative value to particular websites? And do you know where Lawfare falls in that scheme of the general internet?

So I know the general answer to that question, but not the specific answer.

So the general answer is yes, they have very elaborate systems of—at least the good ones have very elaborate systems of— sites and data for quality, and that's so that, you know, Claude and ChatGPT can deliver higher quality information.

The more, the merrier, apparently.

I still have to break 30 links on a single page, but I just don't see that being a problem." So this post doesn't address PageRank dilution, which means the more links you have on a page, the less authority each link gets because the authority is being diluted among links.

But there's this 2012 experiment from Dan Petrovic where he tested if having a back link from a high authority page still passes meaningful ranking value when that page has 4,225 followed outbound links.

And get this, the linked test site jumped from position 79 to number three within about three weeks, which showed that even with thousands of outgoing links, the back link still had a strong ranking effect.

The test is called the PageRank Split Experiment.

And all of this flies directly in the face of what people have been saying with PageRank dilution.

But we're gonna dig into why this might be the case in a little bit when we talk about Google's patent on this.

This Black Hat World post continues, and you're gonna love this one.

It, uh, uh, set it free and let it do its thing.

I lo- I use the analogy of PageRank, which scans billions of web pages to kind of create a, a signal for noise.

That's a completely, uh, complementary model to human intelligence, not replicative.

Uh, the, the fact that we can kind of, uh, create a huge amount and solve legacy problems that we've not been able to solve, I think it's just, it's just fantastic.

Yep, yep.

That was kind of the, the, the PageRank thing that Google did, you know, 20, 20 years ago, to be honest.

[laughs] Sri, I, I love this.

I, I could do this all day long.