`Blog`

Here's a section of the website where I'll just write stuff.
This page is a general index, so you may want to have a look at the other links under blog in the navigation bar.

Too big to succeed I haven't written anything on this blogpost over the last few years. Things have been busy, and this blog was really meant as an explainer of my PhD for my folks. Not that there wasn't a lot to talk about. Let's talk about NLP. …
Wrapping it up Well, it's time to finish up this series of posts. I've covered most of what is in my dissertation. Let's finish things up with a brief recap, shall we? …
Of Squids and Squirrels We've discussed many captivating topics, ranging from distributional substitutability to topological similarity and neural networks. But here's the one topic you really wanted to hear about: squid vs. squirrel. …
Slot Machines Last time on this series of blog posts: Most if not all distributional semantics models can be framed as mathematical estimators of distributional substitutability—to put it plainly, all the vectors we get come from machines that learn how to fill in word slots …
Take me Down to the Paradigm City We need to talk about linguistic structuralism. I swear it won't be as painful as it sounds. Then again, I said the same when I tricked you into math a few posts ago. …
Can a machine write a dictionary? Let's bring out the big guns since the peashooter didn't help…
Your dictionary at a distance Last time (which admittedly was some time ago) we talked about why I wanted to compare embeddings and definitions. This time I'm going to talk about how to do that.…
Comparing Embeddings and Definitions We've talked about word vectors. We've talked about dictionaries. How do we compare them?…
Dictionaries and Semantic Grounding Here's a thought experiment which we owe to Stephen Harnad. Picture an English speaker who doesn't speak Chinese. Give them a Chinese monolingual dictionary and ask them to learn to speak Chinese from that dictionary alone. Take delight in their puzzled look as they wonder what is wrong with you.…
Vectors and Muppets, part II Hail thy muppet and master!…
Vectors and Muppets, part I I used to find physicists' naming sense funny, what's with their OMG particles and dark matter. But now I work in NLP, and the neural nets I work with have names from Sesame Street.…
Gradient Descent into madness I did my best to convince you to stop reading. I discussed dry, nitpicky theoretical arguments to distinguish between two fields that are essentially the same. I made lame duck jokes. I talked about old, dusty, mouldy dictionaries. And yet you chose to remain here. Whatever comes next is on you. Today's topic is calculus. Matrix calculus. …
Dictionaries and NLP If you've read the previous post, then you shouldn't be too surprised that computational linguists don't think of dictionaries as old dusty tomes with no purpose other than fitting the height of their computer screen (well, perhaps some do). …
Some misconceptions about dictionaries And now, something completely different: a six-hundred words lecture on the dictionary…
Counting to meaning Last time, I've talked about distributional semantics in rather vague terms, or what we scientists call "theory". Let's try to get a bit more precise this time around. …
The basics of Distributional Semantics What is the meaning of meaning? I know this question seems borderline smartass-territory, but it has been a major unanswered question in the study of language for a few millennia now. …
Computational Linguistics vs. NLP Have you ever talked to a scientist about their sciencing? They are geeks. They will ramble on, and on, and on about their field of study and never shut up. They will write entire blog posts about their sciencing! …
Computational Linguistics, in a nutshell. Let's play a fun game! I'll give you a type of scientist, and you'll answer with what they do. Well, there's only me here right now, so I'll have to play on my own, but whatever. Game start! …
My PhD, explained to my folks. Every conversation where I try to explain what exactly is my PhD about goes more or less like that: "You know Google translate? Well, NLP is the field of study that cover that kind of applications". And then we swiftly shift topics, for the great relief of everyone involved. …