| View previous topic :: View next topic |
| Author |
Message |
wordcross
|
 Posted: Fri Feb 18, 2011 11:08 pm Post subject: 1 |
 |
|
I thought there was a thread for the IBM Watson Supercomputer playing Jeopardy, but I didn't see it scanning the first couple of pages. So I'll post this rant here:
This computer is *not* better than a human at playing Jeopardy. It *is* better at answering the Jeopardy questions (for the most part), but it was not a fair fight. Watson was fed the questions in text format. For a computer, this is a *lot* simpler than using audio/visual input to create its own data.
If Watson had been required to use speech and text recognition to compete, I would have been much more impressed by a win.
As it stands now, it's still an impressive accomplishment. Getting a computer to understand the variety of natural language and answer in a Jeopardy-style format is no small task. But it doesn't really surprise me. With the advances in computing and linguistic analysis, I'm surprised that this hasn't happened before.
And if I'd been playing against it, the handicap would have pissed me off.
And I haven't seen anyone anywhere even vaguely suggest that this "win" wasn't all it's touted to be, much less challenge the idea that a computer can actually beat a human at Jeopardy. That pisses me off, too.
Not to mention that if there had been any Audio clues, video clues, or picture clues, the humans would have had a much greater advantage, so of course there were not. I don't necessarily blame anyone for that. Jeopardy wanted this to go well so that it would get more publicity, and the same can be said for IBM. Making the machine look foolish for an entire category was not really in anyone's interest. Which is probably a lot of what's fueling this lack of criticism from most places.
Bah.[/rant]
_________________
Has anyone really been far even as decided to use even go want to do look more like? |
|
| Back to top |
|
 |
The Ragin' South Asian
Head Poncho
|
| Posted: Fri Feb 18, 2011 11:31 pm Post subject: 2 |
|
|
| Yeah, I've discussed this a bit with people. Watson won, but mainly on the strength of questions that seemed like if you dumped them in Google, you'd get the answer. I'm sure Google employs some impressive language processing itself, but I thought the point was that Watson was on a whole different level than search. The questions that would have tripped up straight forward search also tripped up Watson. It seemed like perfectly timing its buzzes was Watson's main strength - you could see the frustration on the human's faces early on as they got beat on questions they clearly knew, and as the game went on, things evened out as they started to get the timing down. |
|
| Back to top |
|
|
Sentran
Ray of Sucking Funshine
|
| Posted: Fri Feb 18, 2011 11:49 pm Post subject: 3 |
|
|
I heard it mentioned on a radio program that they felt Watson should have been delayed a few milliseconds before it was allowed to ring in. I remember seeing that it had a manual ringer just like the human contestants, but a computer's reaction speed is always going to be better.
_________________
Sentran
"Speaking of double negatives, I haven't read greylab yet today." - Lifeinmomland |
|
| Back to top |
|
|
Zag
Unintentionally offensive old coot
|
| Posted: Fri Feb 18, 2011 11:57 pm Post subject: 4 |
|
|
| I rather liked Ken Jennings response to that issue. He said (I'm paraphrasing from memory) that timing and speed are one of the advantages computers have, and that he doesn't begrudge his opponents their strengths. |
|
| Back to top |
|
|
wordcross
|
| Posted: Sat Feb 19, 2011 12:02 am Post subject: 5 |
|
|
Oddly enough, from what I understood, the buzzer wasn't actually that big an issue. Watson did have a manual buzzer, and would only buzz in if it had a certain threshold of certainty of its answer. Which means it had to come up with an answer before it rang in. And since the human players were actually listening to the question, they knew when Trebek was finished and when the buzzers were likely to be available. i.e they could anticipate the moment when the buzzers would start working. But if they hit it early, the buzzer has a quarter-second delay before pressing the button will work again.
However, assuming that Watson got the text of the question at the same time that Trebek starts reading a clue, who knows how long it actually takes him to come up with an answer and how it knows when to buzz in. I haven't heard anything about that particular part.
_________________
Has anyone really been far even as decided to use even go want to do look more like? |
|
| Back to top |
|
|
Silversunset
Drama Queen^42
|
| Posted: Sat Feb 19, 2011 12:33 am Post subject: 6 |
|
|
I agree that it wasn't really a fair fight. Even if Watson had the same type of information that humans had (i.e. books and songs rather than simple data tables they way IBM described) a human wouldn't have access to the entire text of a book to reference like Watson would. You couldn't scan your brain for an obscure quote somewhere in the book, you'd just know someone said something like ______ to answer with. a little bit unfair if you ask me.
_________________
as you wish.... |
|
| Back to top |
|
|
wordcross
|
| Posted: Sat Feb 19, 2011 12:50 am Post subject: 7 |
|
|
That's a much more general gripe, though. It's a fairly common argument that computers can do things that humans can do through brute force rather than using human-style logic or learning. Whether or not that's "fair" is up for argument. Personally, I think as long as contestants are given the same input it's a fair fight. A bit lop-sided, sure, but fair.
_________________
Has anyone really been far even as decided to use even go want to do look more like? |
|
| Back to top |
|
|
Thok*
Guest
|
| Posted: Sat Feb 19, 2011 1:30 am Post subject: 8 |
|
|
| wordcross wrote: |
| However, assuming that Watson got the text of the question at the same time that Trebek starts reading a clue, who knows how long it actually takes him to come up with an answer and how it knows when to buzz in. I haven't heard anything about that particular part. |
Length of text roughly equals length of time to ring in. I'm not sure if that's how it worked, but that's a reasonable first approximation. (The other reasonable approximation is that the computer was given some indication whether or not the question was done being read when it was ready to buzz in, and then just waited a bit if it was too early. And before you complain, humans are also given an indication when the question is done being read, because Alex stops talking.) |
|
| Back to top |
|
|
LordKinbote
Daedalian Member
|
| Posted: Sat Feb 19, 2011 3:33 am Post subject: 9 |
|
|
| wordcross wrote: |
| If Watson had been required to use speech and text recognition to compete, I would have been much more impressed by a win. |
I've heard other people say this, and I'm not sure I understand why speech and text recognition would make the win more impressive. Computationally, both the problem of "reading" text and the problem of understanding speech are much, much simpler than the problem of understanding what a question is asking. My phone can read text in many fonts in seconds; I'm sure a supercomputer could do it instantaneously.
The Jeopardy stage is also the best case scenario both for text and speech. The font is fixed and easily read against the blue background. The speech is read clearly and by the same person every time. |
|
| Back to top |
|
|
LordKinbote
Daedalian Member
|
| Posted: Sat Feb 19, 2011 3:39 am Post subject: 10 |
|
|
| The Ragin' South Asian wrote: |
| Yeah, I've discussed this a bit with people. Watson won, but mainly on the strength of questions that seemed like if you dumped them in Google, you'd get the answer. |
What do you mean by "you'd get the answer"? When humans dump a question into Google, we still only receive a list of webpages and then have to do at least a *little* bit of parsing to figure out what words on the page correspond to the question's answer. Google can't give you an answer, it can only tell you where you might look for it. |
|
| Back to top |
|
|
Silversunset
Drama Queen^42
|
| Posted: Sat Feb 19, 2011 3:55 am Post subject: 11 |
|
|
| Quote: |
| It's a fairly common argument that computers can do things that humans can do through brute force rather than using human-style logic or learning. Whether or not that's "fair" is up for argument. Personally, I think as long as contestants are given the same input it's a fair fight. A bit lop-sided, sure, but fair. |
i guess, but it seems unfair because, while the contestants have the same access to the input they do not have the same access to the data in order to answer the question. There isn't really a way to get around that, computers have to have *somewhere* to look for their data but i think it was still an unfair competition to begin with. Rather than a true competition where the contestants have the same perceived data access it seemed more like a test to see if the computer can search faster than a person can think. In that sense, it was impressive, Watson was able to search through all his data in such a split second, but i don't think it was a true competition. more a showcase.
although i will say, I did learn something new from Watson. i never knew Toronto was in the US  Guess it's back to highschool for me ...
_________________
as you wish.... |
|
| Back to top |
|
|
wordcross
|
| Posted: Sat Feb 19, 2011 4:03 am Post subject: 12 |
|
|
| LordKinbote wrote: |
| wordcross wrote: |
| If Watson had been required to use speech and text recognition to compete, I would have been much more impressed by a win. |
I've heard other people say this, and I'm not sure I understand why speech and text recognition would make the win more impressive. Computationally, both the problem of "reading" text and the problem of understanding speech are much, much simpler than the problem of understanding what a question is asking. My phone can read text in many fonts in seconds; I'm sure a supercomputer could do it instantaneously.
The Jeopardy stage is also the best case scenario both for text and speech. The font is fixed and easily read against the blue background. The speech is read clearly and by the same person every time. |
And yet they chose not to. It just adds that much more to the difficulty. For all that it has come a long way, text and especially speech recognition are still not perfect. If we want a computer to challenge humans in the things we are best at (such as language) then to claim that the computer is "better" than humans it must be on an even playing field.
Like i said, I recognize that this is a great leap forward in linguistic computation, and I don't think that Watson is by any means insignificant. It just annoys me that they're treating this as though Watson is as good at playing jeopardy as a human is. Maybe he is, but until they can put him through the same data input procedures as the human players get, it's apples to oranges.
_________________
Has anyone really been far even as decided to use even go want to do look more like? |
|
| Back to top |
|
|
Chaz
Vote: Zag
|
| Posted: Sat Feb 19, 2011 4:21 am Post subject: 13 |
|
|
| LordKinbote wrote: |
| What do you mean by "you'd get the answer"? When humans dump a question into Google, we still only receive a list of webpages and then have to do at least a *little* bit of parsing to figure out what words on the page correspond to the question's answer. Google can't give you an answer, it can only tell you where you might look for it. |
That's not entirely true. Google answers a lot of questions straight out.
How many people live in the US
What was amazing is the deduction of a clue to a question.
For example, if I say "a global network of computers," most people would know exactly what I was talking about; however, somebody who didn't wouldn't be able to google the answer (even if they did have the Internet.)
_________________
The enemy's base is down. |
|
| Back to top |
|
|
LordKinbote
Daedalian Member
|
| Posted: Sat Feb 19, 2011 5:07 am Post subject: 14 |
|
|
| Chaz wrote: |
| LordKinbote wrote: |
| What do you mean by "you'd get the answer"? When humans dump a question into Google, we still only receive a list of webpages and then have to do at least a *little* bit of parsing to figure out what words on the page correspond to the question's answer. Google can't give you an answer, it can only tell you where you might look for it. |
That's not entirely true. Google answers a lot of questions straight out.
How many people live in the US |
I guess I so rarely actually enter a question into the Google search engine that I didn't realize how much you could get back. Of course, Watson does not have access to the internet so it isn't quite able to use Google's method. |
|
| Back to top |
|
|
Chaz
Vote: Zag
|
| Posted: Sat Feb 19, 2011 5:28 am Post subject: 15 |
|
|
I wonder how many times this conversation already happened when Deep Blue beat Kasparov.
_________________
The enemy's base is down. |
|
| Back to top |
|
|
LordKinbote
Daedalian Member
|
| Posted: Sat Feb 19, 2011 5:44 am Post subject: 16 |
|
|
| Chaz wrote: |
| I wonder how many times this conversation already happened when Deep Blue beat Kasparov. |
The only thing they added to Deep Blue between 1996 and 1997 was a death ray. |
|
| Back to top |
|
|
Chaz
Vote: Zag
|
| Posted: Sat Feb 19, 2011 3:44 pm Post subject: 17 |
|
|
_________________
The enemy's base is down. |
|
| Back to top |
|
|
extro...*
Guest
|
| Posted: Sat Feb 19, 2011 3:47 pm Post subject: 18 |
|
|
| wordcross wrote: |
| LordKinbote wrote: |
| wordcross wrote: |
| If Watson had been required to use speech and text recognition to compete, I would have been much more impressed by a win. |
I've heard other people say this, and I'm not sure I understand why speech and text recognition would make the win more impressive. Computationally, both the problem of "reading" text and the problem of understanding speech are much, much simpler than the problem of understanding what a question is asking. My phone can read text in many fonts in seconds; I'm sure a supercomputer could do it instantaneously.
The Jeopardy stage is also the best case scenario both for text and speech. The font is fixed and easily read against the blue background. The speech is read clearly and by the same person every time. |
And yet they chose not to. It just adds that much more to the difficulty. For all that it has come a long way, text and especially speech recognition are still not perfect. If we want a computer to challenge humans in the things we are best at (such as language) then to claim that the computer is "better" than humans it must be on an even playing field.
Like i said, I recognize that this is a great leap forward in linguistic computation, and I don't think that Watson is by any means insignificant. It just annoys me that they're treating this as though Watson is as good at playing jeopardy as a human is. Maybe he is, but until they can put him through the same data input procedures as the human players get, it's apples to oranges. |
I agree with LordKinbote completely on this one. There are iPhone apps that can do OCR (optical character recognition), and as pointed out, the text on Jeopardy is so clear and consistent that recognition would be flawless and virtually instantaneous. There isn't any question it could have been done that way, and I think it wasn't done that way because it would be understood it doesn't prove anything that hasn't already been proven. There are speech to text programs too, that require no training. Such could likely have been made flawless by training on recordings of Trebek's voice, but given Watson could have read the text, asking for hearing the spoken words would be like complaining that a deaf human contestant has an advantage because he's only reading the text, and not hearing words.
There's no question Watson could have done as well without having been given electronic text input, by simply augmenting Watson with a camera to see and some ordinary OCR software.
There will always be claims of an unfair advantage when a machine is made to beat humans at some task. |
|
| Back to top |
|
|
wordcross
|
| Posted: Sat Feb 19, 2011 4:14 pm Post subject: 19 |
|
|
| extro...* wrote: |
| There will always be claims of an unfair advantage when a machine is made to beat humans at some task. |
But my gripe is not the same as SS's who says it's unfair because the computer has perfect recall. That's what the computer was made for. It's the nature of the computer to have that. I have no problem with contests where the abilities are obviously skewed in one direction. My issue is when the methodology for gameplay is different for one or more players.
Whether or not Watson *could have* done it is not the issue. Getting a text file is different than using some of that processing power to parse visual/speech data into something the computer understands. Maybe it's a miniscule difference that would have resulted in a millisecond of delay, but sometimes that millisecond can make a difference (as the argument over the buzzers indicated). And since the recognition software is not perfect, the rare mistakes that *do* happen could also make a difference. I'm not saying that Watson couldn't have done it, or wouldn't still have won, or that he's not still amazing, but in any case for anything ever, if the gameplay is not the same for each player, it is inherently unfair.
_________________
Has anyone really been far even as decided to use even go want to do look more like? |
|
| Back to top |
|
|
Thok
Oh, foe, the cursed teeth!
|
| Posted: Sat Feb 19, 2011 4:51 pm Post subject: 20 |
|
|
| That people are less impressed by Watson then by Deep Blue says more about how accustomed we've gotten to computers than the relative difficulty of the task. |
|
| Back to top |
|
|
Courk
Daedalian Member
|
| Posted: Sat Feb 19, 2011 5:09 pm Post subject: 21 |
|
|
| I forgot to record this! Aaahhhrgh! |
|
| Back to top |
|
|
wordcross
|
| Posted: Sat Feb 19, 2011 5:13 pm Post subject: 22 |
|
|
| Thok wrote: |
| That people are less impressed by Watson then by Deep Blue says more about how accustomed we've gotten to computers than the relative difficulty of the task. |
That is very true. As extro and LK have both mentioned, we have smartphones that do text and speech recognition, and we have near instantaneous access to more information than has ever been available before. And it's all thanks to computers. It does not seem like such a huge leap that a computer should be able to excel at trivia.
Of course, for IBM, that's not what this is actually about. It's about recognizing natural language. At that task Watson has excelled beyond expectation, and spectacularly.
_________________
Has anyone really been far even as decided to use even go want to do look more like? |
|
| Back to top |
|
|
extro...*
Guest
|
| Posted: Sat Feb 19, 2011 11:08 pm Post subject: 23 |
|
|
| wordcross wrote: |
| Whether or not Watson *could have* done it is not the issue. Getting a text file is different than using some of that processing power to parse visual/speech data into something the computer understands. |
They wouldn't have had to use some of the existing processing power - they could have added more. A computer on the front end with a camera pointed at the game screen, and a network cable back to Watson to feed it the ascii text.
| wordcross wrote: |
| Maybe it's a miniscule difference that would have resulted in a millisecond of delay, but sometimes that millisecond can make a difference (as the argument over the buzzers indicated). |
I haven't been able to find enough details as to the timing of how the text was fed to Watson. Did they send it the entire text the very instant the written question appeared on screen for the human players, and before Trebek even began reading the question?
It seems to me using an OCR approach would have given Watson a distinct further advantage, in that he would have the ascii text of the question nearly the instant it was displayed, well before the human players finished reading the first few words.
| wordcross wrote: |
| And since the recognition software is not perfect, the rare mistakes that *do* happen could also make a difference. |
In the context of Jeopardy, OCR software could be made to work perfectly.
| wordcross wrote: |
| I'm not saying that Watson couldn't have done it, or wouldn't still have won, or that he's not still amazing, but in any case for anything ever, if the gameplay is not the same for each player, it is inherently unfair. |
So they should have made it fair by handing each human player the end of an ethernet cable, over which the text was being transmitted, to use as they saw fit. |
|
| Back to top |
|
|
wordcross
|
| Posted: Sat Feb 19, 2011 11:25 pm Post subject: 24 |
|
|
Then they're not playing jeopardy.
_________________
Has anyone really been far even as decided to use even go want to do look more like? |
|
| Back to top |
|
|
MatthewV
Daedalian Member :_
|
| Posted: Sat Feb 19, 2011 11:39 pm Post subject: 25 |
|
|
If I recall from my flawed memory and without doing random internet research again, this feat is much more similar to when a computer beat the best human backgammon player. This happened sometime in the 1970's. People complained about the computer getting better dice rolls (there is even a published paper showing this was the case) and yadda yadda.
Now, when a computer beats a top ranked human at go, I will be impressed. |
|
| Back to top |
|
|
lexprod
NOT not a title
|
| Posted: Sat Feb 19, 2011 11:53 pm Post subject: 26 |
|
|
When a computer wins The Bachelor, fooling humans into loving it, then I will be impressed 
As for buzzer timing, this is a bit of a "random fact I thought I heard," but I thought there was a series of lights under the clue board that lit up to signal the players when buzzing can begin? Otherwise it'd just be guessing at when the buzzer operator decides Alex is done talking right?
_________________
I'll have a P please... |
|
| Back to top |
|
|
Zag
Unintentionally offensive old coot
|
| Posted: Sun Feb 20, 2011 12:05 am Post subject: 27 |
|
|
| Go, like chess, is inevitable. They both have fixed rules and a finite (if extremely large) solution tree. Go is several orders of magnitude more difficult, but throw enough computing power at it and it can be solved just as Checkers already has been. Solving Chess is the same order of magnitude as the work that has been done so far on the human genome project. I'm not sure how much more solving Go would be, but the world computing power continues to increase, and it will fall eventually. |
|
| Back to top |
|
|
Chaz
Vote: Zag
|
| Posted: Sun Feb 20, 2011 2:00 am Post subject: 28 |
|
|
I agree with extro. If anything, the "feed" Watson got was a delay so the computer couldn't "understand" the question before the human players did. OCR would have given Watson the question seconds before the human players could have even started to process it.
Again, the impressive part isn't even that the computer was able to answer the questions. The impressive part was that it could understand what the question meant, and give the reply in a form of a question.
| Quote: |
| When a computer wins The Bachelor, fooling humans into loving it, then I will be impressed. |
Hasn't that already happened?
http://jezebel.com/#!5331003/woman-to-wed-amusement-park-ride
I'll be impressed when a computer can win a simple game of jacks.
_________________
The enemy's base is down. |
|
| Back to top |
|
|
MNOWAX
0.999... of a Troll
|
| Posted: Sun Feb 20, 2011 3:15 am Post subject: 29 |
|
|
I remember her! the link doesnt' work, but she wanted to marry that lifting Rainbow ride and made out with it on camera shown a BBC special. They ACTUALLY got her and another "objective sexual" together to trade love stories about the Berlin wall, and other objects. Apparently Objects and Objective sexuals are polyamorus.  .
_________________
The Man The Myth The Legend
MNOWAX |
|
| Back to top |
|
|
MNOWAX
0.999... of a Troll
|
|
| Back to top |
|
|
MNOWAX
0.999... of a Troll
|
| Posted: Sun Feb 20, 2011 3:25 am Post subject: 31 |
|
|
I also want to point out that Watson did the worst on the category that had to refer to the answer category to come up with a correct response. For a fake example: in the category "Also a Brand Name" with a clue "a person running fast", Watson couldn't come up with "What is Sprint" because i don't believe it was remembering the category it was in. Or coming up with Toronto in the first Final jeopardy answer in the category US cities.
_________________
The Man The Myth The Legend
MNOWAX |
|
| Back to top |
|
|
The Ragin' South Asian
Head Poncho
|
| Posted: Sun Feb 20, 2011 4:28 am Post subject: 32 |
|
|
| They talked about that wrt the Toronto answer. They gave the categories a weak weighting relative to the question text, which I guess worked better overall but lead to a few major whiffs. |
|
| Back to top |
|
|
Chaz
Vote: Zag
|
| Posted: Sun Feb 20, 2011 5:33 am Post subject: 33 |
|
|
For anyone who knows how to use torrents (the legality of which is a grey area, but this is the grey labyrinth, so...)
Part 1 Part 2 Part 3
For those of you who don't, you'll need a torrent client. I'd suggest uTorrent.
_________________
The enemy's base is down. |
|
| Back to top |
|
|
extro...*
Guest
|
| Posted: Sun Feb 20, 2011 5:48 am Post subject: 34 |
|
|
| Regarding the Toronto answer, I can't help wondering if the programming didn't have a problem and confused things with the names of things. For instance there is a Toronto, Ohio ... so that makes Toronto a U.S. city. Then if it found airports in Toronto, Ontario that maybe fit the criteria, since Toronto is a U.S. city (a different Toronto), it guessed Toronto. |
|
| Back to top |
|
|
Chaz
Vote: Zag
|
| Posted: Sun Feb 20, 2011 7:26 pm Post subject: 35 |
|
|
I think it was an intentional mistake to make people laugh.
What I was curious about was the amount that Watson would risk.
Was it based on how sure Watson was about his guess?
_________________
The enemy's base is down. |
|
| Back to top |
|
|
Chuck
Daedalian Member
|
| Posted: Sun Feb 20, 2011 7:47 pm Post subject: 36 |
|
|
| Does it know the other players' scores so it can bet strategically? |
|
| Back to top |
|
|
Zag
Unintentionally offensive old coot
|
| Posted: Sun Feb 20, 2011 9:18 pm Post subject: 37 |
|
|
| I'm sure it does. |
|
| Back to top |
|
|
The Ragin' South Asian
Head Poncho
|
|
| Back to top |
|
|
Samadhi
+1
|
| Posted: Sun Feb 20, 2011 11:54 pm Post subject: 39 |
|
|
_________________
And he lived happily ever after. Except for the dieing at the end and the heartbreak in between. |
|
| Back to top |
|
|
MatthewV
Daedalian Member :_
|
| Posted: Mon Feb 21, 2011 12:30 am Post subject: 40 |
|
|
| Chaz wrote: |
| For anyone who knows how to use torrents (the legality of which is a grey area, but this is the grey labyrinth, so...) |
For those people at universities or with ISP that shut you down, generously seed old movies and your ratio will stay high and you won't get caught. |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
