So much headdesk....

[davidgillon]

Toyota pauses Paralympics self-driving buses after one hits visually impaired athlete

"Tokyo police said that vehicle operators had told them they “were aware that a person was there but thought [the person] would [realize that a bus was coming] and stop crossing the [street]”, according to the Asahi Shimbun newspaper."

Absolutely classic case of ableds assuming disabled people will react in exactly the way they would. Not only did the designers of the self-driving system fail to consider it, but so did the safety monitors on the spot.

It's like the 'shared space' notion of drivers and pedestrians catching each others' eyes to negotiate who goes first, which completely failed to wonder "So how will a person with a visual impairment do that?"

I'm absolutely convinced we're going to have cases of autonomous self-driving vehicles hitting disabled people because we didn't trigger the human recognition functionality of its synthetic vision system - I've seen footage of an AI supposed to edge-detect people in a crowd and draw an outline around them that consistently every rejected parent with a buggy and I've absolutely no doubt would have treated wheelchair and scooter users identically, and possibly ambulatory disableds with mobility aids as well

Comments

[sovay]

Paralympics self-driving buses after one hits visually impaired athlete

Oh, blech on all levels.

[cosmolinguist]

Even without allowing for visually impaired people, self-driving cars should not be assuming pedestrians who see cars will stop for them because that has never been my experience!

Actually waiting for a pedestrian crossing light has made me stand out so much that once someone crossed the road against the light, saw me still waiting, crossed back to me, and got right up in my face to say "it's safe to cross!" That's how weird I am for waiting for safe(r) crossing times.

[davidgillon]

My driving instructor would have been extremely unimpressed if I'd failed to treat a pedestrian as a risk and drive accordingly, no matter what they were actually doing - he actually used to make me call out every potential risk I could see as we were driving along.

[capri0mni]

I'm absolutely convinced we're going to have cases of autonomous self-driving vehicles hitting disabled people because we didn't trigger the human recognition functionality of its synthetic vision system

And I'm convinced that (at least) half the time, we already don't "trigger human recognition functionality" of human wetware, either, based on how rarely people have made room for me in crowded hallways, or navigating aisles between tables in restaurants (but have no trouble making room for an ambulatory kid, whose head is level with mine, in-chair).

[davidgillon]

Absolutely! I was really struck by a similar point John Scalzi made in Lock In about how people fail to treat the full-body prosthetics being used by people with Haden's Syndrome (the backstory's infectious version of Locked In Syndrome) as people and constantly get in their way in crowds. For a non-wheelchair user like Scalzi that was particularly insightful.

[capri0mni]

That was insightful! It's nice to have confirmation that a spec-fic writer is using their imagination for more than just fancy plot twists.

[jesse_the_k]

I like to think that Scalzi was smart enough to get a "sensitivity reader" -- a practice I strongly endorse, even though I hate the name. The issue isn't "sensitivity," it's cultural respect.

[davidgillon]

He seemed pretty relieved when I mentioned how it tallied with wheelie experience, though that just mean his sensitivity reader wasn't a wheelie.

I've actually done a couple of sensitivity reads myself.

[jesse_the_k]

It's terrifying how gleefully the young and nondisabled embrace self-driving cars. (Yes, there is a use case for disabled drivers, but I want to live in a world where adulthood doesn't require a drivers license.)

The US federal access folks are on it.

[davidgillon]

Interesting! Tab opened so I can come back and give that the attention it deserves.