I spent 10% of my life contributing to the development of the #VisionPro while I worked at Apple as a Neurotechnology Prototyping Researcher in the Technology Development Group. It’s the longest I’ve ever worked on a single effort. I’m proud and relieved that it’s finally announced. I’ve been working on AR and VR for ten years, and in many ways, this is a culmination of the whole industry into a single product. I’m thankful I helped make it real, and I’m open to consulting and taking calls if you’re looking to enter the space or refine your strategy.
The work I did supported the foundational development of Vision Pro, the mindfulness experiences, ▇▇▇▇▇▇ products, and also more ambitious moonshot research with neurotechnology. Like, predicting you’ll click on something before you do, basically mind reading. I was there for 3.5 years and left at the end of 2021, so I’m excited to experience how the last two years brought everything together. I’m really curious what made the cut and what will be released later on.
Specifically, I’m proud of contributing to the initial vision, strategy and direction of the ▇▇▇▇▇▇ program for Vision Pro. The work I did on a small team helped green light that product category, and I think it could have significant global impact one day.
The large majority of work I did at Apple is under NDA, and was spread across a wide range of topics and approaches. But a few things have become public through patents which I can cite and paraphrase below.
Generally as a whole, a lot of the work I did involved detecting the mental state of users based on data from their body and brain when they were in immersive experiences.
So, a user is in a mixed reality or virtual reality experience, and AI models are trying to predict if you are feeling curious, mind wandering, scared, paying attention, remembering a past experience, or some other cognitive state. And these may be inferred through measurements like eye tracking, electrical activity in the brain, heart beats and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance etc.
There were a lot of tricks involved to make specific predictions possible, which the handful of patents I’m named on go into detail about. One of the coolest results involved predicting a user was going to click on something before they actually did. That was a ton of work and something I’m proud of. Your pupil reacts before you click in part because you expect something will happen after you click. So you can create biofeedback with a user's brain by monitoring their eye behavior, and redesigning the UI in real time to create more of this anticipatory pupil response. It’s a crude brain computer interface via the eyes, but very cool. And I’d take that over invasive brain surgery any day.
Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it.
Another patent goes into details about using machine learning and signals from the body and brain to predict how focused, or relaxed you are, or how well you are learning. And then updating virtual environments to enhance those states. So, imagine an adaptive immersive environment that helps you learn, or work, or relax by changing what you’re seeing and hearing in the background.
All of these details are publicly available in patents, and were carefully written to not leak anything. There was a ton of other stuff I was involved with, and hopefully more of it will see the light of day eventually.
A lot of people have waited a long time for this product. But it’s still one step forward on the road to VR. And it’s going to take until the end of this decade for the industry to fully catch up to the grand vision for this tech.
Again, I’m open to consulting work and taking calls if your business is looking to enter the space or refine your strategy. Mostly, I’m proud and relieved this has finally been announced. It’s been over five years since I started working on this, and I spent a significant portion of my life on it, as did an army of other designers and engineers. I hope the whole is greater than the sum of the parts and Vision Pro blows your mind.
Imagine being @JoeBiden. Your son makes millions of dollars from foreign entities but does not pay taxes because that would mean disclosing that he is engaged in foreign lobbying. With the millions of dollars and no serious job, he becomes a drug addict and illegally obtains a gun. All he tells them is that he is the son of the US Vice President, and he claims that part of the money will go to the "Big Guy." He promises them access to his dad and top US officials. They hire him without any qualifications and pay him millions of dollars.
Eventually, the information comes out, and the Justice Department has to investigate. The investigation drags on for five years, and eventually, they strike a deal with him to plead guilty to two misdemeanors of tax evasion so that the government would not prosecute him for illegally possessing a gun as a drug addict, a serious crime that has sent tens of thousands of Americans to prison for years.
Imagine being the same father who, as a senator, drafted a draconian anti-drug law that sent tens of thousands of black people and others to prison, confiscated all their belongings, and gave a big speech boasting that the drug addicts should all be taken off the streets. Now that your son is one of them, when you're asked about his case, you say you're proud of him and he has done nothing wrong.
Imagine journalists and commentators like @joenbc justifying it on television, singing your praises when their own child would have been in prison long ago for the same crimes. Imagine the people who believed in the rule of law watching the two-tiered justice system so brazenly displayed.
America is in crisis. May God help us all!
LEARN MORE ABOUT CHD’s EMR + WIRELESS PROGRAM 👇
Through litigation, education and advocacy, we strive to empower individuals to create environments that are conducive to humans, with the best interests of children, health + the environment in mind.
https://t.co/QalzTt8Q17
While we struggle to afford food and fuel, the elite gathered at Davos/WEF to discuss how we should be allowed to live our lives. Their totalitarian dream of a so-called “Great Reset” where people own nothing and are tracked/monitored with every step, must be condemned by us all.
I’d already deleted Instagram and FB from my phone last year...Might as well make it all three!
Parler: @ train4life
MeWe: https://t.co/lzg5ViSaT7
Minds: @ train4life
...or Telegram group: https://t.co/OOc13XQm72
Pharmaceutical interventions are not political. By conflating corrupt toxic industries with political rivalry we lose the focus which is our own human body. It’s not only incredibly thick to conflate health with political identity but also damaging. https://t.co/PluQPay4j3
Directly from HHS data, the flu vaccine is the highest compensated vaccine for adverse events through the vaccine court paying out over $1B https://t.co/WUlTcCOm0J
Brandy Vaughan Found Dead>>>
https://t.co/qPAxDuqBWW
Brandy Vaughn passed away at the age of 47 on Tuesday, December 8, 2020, with friend and families left in heartbreak and pains.
++++++++++++
https://t.co/pBfOzIBlEM
Politicians who enforce lockdowns should not be paid their salaries until lockdowns are ended.
They've forced millions to shut their businesses and millions more into unemployment. Easy to do because it doesn't hurt their livelihood.
I think this is a perfectly fair suggestion.
EXACTLY! Between this and “warspeed” 5G rollout executive order. And hasn’t everyone figured out that our government doesn’t actually run our country yet?!
Lots of journalists insisting there's only one valid way to do journalism - submitting to the top-down structure of corporate outlets: editors are the decision-makers & bosses - even as their profession collapses around them & they have nothing to point to in their work of note.
BREAKING: Photos have emerged of California's hypocritical Gov. Gavin Newsom at his outrageous Napa dinner extravaganza: Loud, obnoxious, no masks, maybe not even outdoors! RT via @FoxLA
https://t.co/oT1dhJqxuM