Christoph Schneider

I like products, gadgets, UX and the people and organizations that make them great. All opinions are strictly my own.

[Luke Wroblewski @LukeW]

but why?
https://t.co/Tyc5hhGZvA

[Christoph Schneider @cschnd]

@johncutlefish But the other person also needs to be in a hallway.

[Christoph Schneider @cschnd]

@RazorUX @NikiasMolina Nailed it

[Christoph Schneider @cschnd]

@sklyarov @zollotech You could use guided access to block touches. It lets you define areas of the screen that are blocked.

[Christoph Schneider @cschnd]

@A_i Ted Chiang and Outer Wilds in one tweet, who would have thought 💛

[Christoph Schneider @cschnd]

@jbrooksuk @bilalsattar @zollotech I had to restart the phone after all processing was done, then it showed up as voice for Text-To-Speech

[Christoph Schneider @cschnd]

@stdwanze Folg ihm halt - Galaxy Brain Problem Solved

[Christoph Schneider @cschnd]

@tzijlstra I tried exactly one workout a few weeks back and was still unimpressed. Not offering e.g. core workouts for 3 different levels is a mistake in my opinion

[Christoph Schneider @cschnd]

@tzijlstra Yeah, could not have gone much worse

[Christoph Schneider @cschnd]

Sad to see Reddit go down the drain, it will become unusable for me after this month without any replacement in sight

[Sebastiaan de With @sdw]

Reddit’s AMA with its CEO on their API (read: third party app killing) is a train wreck. These are remakes from their CEO. Why does Reddit consistently have such terrible leadership?

Before this, Huffman had no public interactions with the community or website for 10 months.

[Christoph Schneider @cschnd]

@MKBHD Maybe it’s a great cinematic shot which they have copied, not any meaning or message.

[Christoph Schneider @cschnd]

@tzijlstra I don’t think there will be one killer application, same as with e.g. the iPad. For more and more things it will just be better to use than traditional devices, and for everyone there is a threshold when it makes sense to get one.

[Christoph Schneider @cschnd]

Huh, Apple might actually serious about a gaming push this time. Valve has shown how it’s done with the ⁦@OnDeck⁩ https://t.co/kroOXQiY9d

[Christoph Schneider @cschnd]

@tzijlstra Interesting take! I think overall strain on eyes and head will be in the way of a full workday replacement, and the value over normal monitors is not that high. But it is a nice extra use case, especially in the transition where 2D apps in 3D space bridge a gap.

[Christoph Schneider @cschnd]

Fantastic read. More innovation, more dystopia.

[Sterling Crispin 🕊️ @sterlingcrispin]

I spent 10% of my life contributing to the development of the #VisionPro while I worked at Apple as a Neurotechnology Prototyping Researcher in the Technology Development Group. It’s the longest I’ve ever worked on a single effort. I’m proud and relieved that it’s finally announced. I’ve been working on AR and VR for ten years, and in many ways, this is a culmination of the whole industry into a single product. I’m thankful I helped make it real, and I’m open to consulting and taking calls if you’re looking to enter the space or refine your strategy.

The work I did supported the foundational development of Vision Pro, the mindfulness experiences, ▇▇▇▇▇▇ products, and also more ambitious moonshot research with neurotechnology. Like, predicting you’ll click on something before you do, basically mind reading. I was there for 3.5 years and left at the end of 2021, so I’m excited to experience how the last two years brought everything together. I’m really curious what made the cut and what will be released later on.

Specifically, I’m proud of contributing to the initial vision, strategy and direction of the ▇▇▇▇▇▇ program for Vision Pro. The work I did on a small team helped green light that product category, and I think it could have significant global impact one day.

The large majority of work I did at Apple is under NDA, and was spread across a wide range of topics and approaches. But a few things have become public through patents which I can cite and paraphrase below.

Generally as a whole, a lot of the work I did involved detecting the mental state of users based on data from their body and brain when they were in immersive experiences.

So, a user is in a mixed reality or virtual reality experience, and AI models are trying to predict if you are feeling curious, mind wandering, scared, paying attention, remembering a past experience, or some other cognitive state. And these may be inferred through measurements like eye tracking, electrical activity in the brain, heart beats and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance etc.

There were a lot of tricks involved to make specific predictions possible, which the handful of patents I’m named on go into detail about. One of the coolest results involved predicting a user was going to click on something before they actually did. That was a ton of work and something I’m proud of. Your pupil reacts before you click in part because you expect something will happen after you click. So you can create biofeedback with a user's brain by monitoring their eye behavior, and redesigning the UI in real time to create more of this anticipatory pupil response. It’s a crude brain computer interface via the eyes, but very cool. And I’d take that over invasive brain surgery any day.

Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it.

Another patent goes into details about using machine learning and signals from the body and brain to predict how focused, or relaxed you are, or how well you are learning. And then updating virtual environments to enhance those states. So, imagine an adaptive immersive environment that helps you learn, or work, or relax by changing what you’re seeing and hearing in the background.

All of these details are publicly available in patents, and were carefully written to not leak anything. There was a ton of other stuff I was involved with, and hopefully more of it will see the light of day eventually.

A lot of people have waited a long time for this product. But it’s still one step forward on the road to VR. And it’s going to take until the end of this decade for the industry to fully catch up to the grand vision for this tech.

Again, I’m open to consulting work and taking calls if your business is looking to enter the space or refine your strategy. Mostly, I’m proud and relieved this has finally been announced. It’s been over five years since I started working on this, and I spent a significant portion of my life on it, as did an army of other designers and engineers. I hope the whole is greater than the sum of the parts and Vision Pro blows your mind.

[Christoph Schneider @cschnd]

It’s the price Apple paid for focus. Descope as much as possible to get Vision Pro market ready.

[Frank Thelen @frank_thelen]

I really think this must be a joke #iOS17 #macossonoma especially with one of the biggest R&D budgets in the world #WWDC23

[Christoph Schneider @cschnd]

I had not thought about that yet: even those who got a personal demo never saw someone else wearing it.

The demo units probably did not have EyeSight yet.

[Zoe Kleinman @zsk]

Yesterday I tried the Apple Vision Pro. I wasn't allowed to film or photograph it. It was in a private room at Apple Park. I have yet to see *anyone* outside of the promo ad actually wearing one. I have tried a lot of headsets! here's what I thought of this one....

[Christoph Schneider @cschnd]

EyeSight has strong watchOS 1 UI vibes. It will either be drastically improved in 1-2 iterations or completely gone. My guess is on the latter.

[Christoph Schneider @cschnd]

(2/2) dystopian consequences are much more obvious.

[Christoph Schneider @cschnd]

(1/2) Couldn’t stop thinking yesterday about the possibilities and consequences of a headset that is meant to be constantly worn like the Apple Vision Pro.

It feels similar to how smartphones have changed almost everything about our daily lives, except that this time the