holding it to pieces

[Sterling Crispin 🕊️ @sterlingcrispin]

Vision Pro mega-thread 1/5:

My advice for designing and developing products for Vision Pro. This thread includes a basic overview of the platform, tools, porting apps, general product design, prototyping, perceptual design, business advice and more.

Disclaimer: I’m not an Apple representative. This is my personal opinion and does not contain non-public information.

Overview:

Apps on visionOS are organized into “scenes”, which are Windows, Volumes, and Spaces.

Windows are a spatial version of what you’d see on a normal computer. They’re bounded rectangles of content that users surround themselves with. These may be windows from different apps or multiple windows from one app.

Volumes are things like 3D objects, or small interactive scenes. Like a 3D map, or small game that’s not immersive.

Spaces are fully immersive experiences where only one app is visible. That could be full of many Windows and Volumes from your app. Or like VR games where the system goes away and it's all custom content. You can think of visionOS itself like a Shared Space where apps coexist together and you have less control. Whereas Full Spaces give you the most control and immersiveness, but don’t coexist with other apps. Spaces have immersion styles: mixed, progressive, and full. Which defines how much or little of the real world you want the user to see.

User Input:

Users can look at the UI and pinch like the demo videos show. But you can also reach out and tap on windows directly, sort of like it’s actually a floating iPad. Or use a bluetooth trackpad or video game controller. You can also look and speak in search bars, but that’s disabled by default for some reason on existing iPad and iOS apps running on Vision Pro. There’s also a Dwell Control for eyes-only input, but that’s really an accessibility feature. For a simple dev approach, your app can just use events like a TapGesture. In this case, you won't need to worry about where these events originate from.

Spatial Audio:

Vision Pro has an advanced spatial audio system that makes sounds seem like they’re really in the room by considering the size and materials in your room. Using subtle sounds for UI interaction and taking advantage of sound design for immersive experiences is going to be really important. Make sure to take this topic seriously.

Development:

If you want to build something that works between Vision Pro, iPad, and iOS, you'll be operating within the Apple dev ecosystem, using tools like XCode and SwiftUI. However, if your goal is to create a fully immersive VR experience for Vision Pro that also works on other headsets like Meta's Quest or PlayStation VR, you have to use Unity.

Apple Tools:

For Apple’s ecosystem, you’ll use SwiftUI to create the UI the user sees and the overall content of your app. RealityKit is the 3D rendering engine that handles materials, 3D objects, and light simulations. You’ll use ARKit for advanced scene understanding. Like if you want someone to throw virtual darts and have them collide with their real wall, or do advanced things with hand tracking. But those rich AR features are only available in Full Spaces. There’s also Reality Composer Pro which is a 3D content editor that lets you drag things around a 3D scene and make media rich Spaces or Volumes. It’s like Diet-Unity that’s built specifically for this development stack.

One cool thing with Reality Composer is that it’s already full of assets, materials, and animations. That helps developers who aren’t artists build something quickly and should help to create a more unified look and feel to everything built with the tool. Pros and cons to that product decision, but overall it should be helpful.

Existing iOS Apps:

If you're bringing an iPad or iOS app over, it will probably work unmodified as a Window in the Shared Space. If your app supports both iPad and iPhone, it’ll look like the iPad version.

You can use the Ornament API to make little floating islands of UI in front of, or besides your app, to make it feel more spatial. But that’s not something all existing apps get automatically. Ironically, if your app is using a lot of ARKit features, you’ll likely need to ‘reimagine’ it significantly as ARKit has been upgraded a lot.

If you’re excited about building something new for Vision Pro, my personal opinion is that you should prioritize how your app will provide value across iPad and iOS too. Otherwise you're losing out on hundreds of millions of users.

Unity:

You can build to Vision Pro with the Unity game engine, which is a massive topic. Again, you need to use Unity if you’re building to Vision Pro as well as a Meta headset like the Quest or PSVR.

Unity supports building Bounded Volumes for the Shared Space which exist alongside native Vision Pro content. And Unbounded Volumes, for immersive content that may leverage advanced AR features. Finally you can also build more VR-like apps which give you more control over rendering but seem to lack support for AR Kit scene understanding like plane detection. The Volume approach gives RealityKit more control over rendering, so you have to use Unity’s PolySpatial tool to convert materials, shaders, and other features.

Unity support for Vision Pro allows for tons of interactions you’d expect to see in VR, like teleporting to a new location or picking up and throwing virtual objects.

[holding it to pieces @_jafon_]

Hi. Not here lately as quality tweets are min. Where’d people go besides big Mastodon? Preferably w/out massive crypto/NFT shill spam. #Twittershutdown #TwitterMigration

[holding it to pieces @_jafon_]

@wildacres_ie @IrishRainforest @Irishwildlife Any pics and info available on what a combination looks like?

[Anya Asano 🔮 @ayenwhyay]

The Behenian Flock
6 rotating astrological gif objects
@foundation
.169 reserve

🌟Algol
🌟Antares
🌟Capella
🌟Procyon
🌟Sirius
🌟Vega

Thanks @manifoldxyz for your 🛠

https://t.co/QqBwgoG9pI

[Anya Asano 🔮 @ayenwhyay]

more rotating heads

[Alex Laplaza 🫠 @laplza]

Stunning work by Dillon Marsh visualizing mining outputs and the mines they came from.

Here's copper.

[spencer 🦈 @Unpop_Science]

WHAT KILLED THE CRABS?

The official story is something like “a billion snow crabs disappeared.”

If that sounds fishy to you, keep reading. Let’s dive into the ecology, oceanography, & geopolitical history of the Bering Sea.

A science thread on crabs, corruption, & collapse: 🦀

[Neil deGrasse Tyson @neiltyson]

Blue Lives Matter.

[Michael Harriot @michaelharriot]

To be fair, it's possible that rapping shoe salesman & his white friends have a better plan for Black people than all the scholars, activists & people who read books. He probably would've told us the plan if he had access to a platform that allows him to explain himself in detail

[Ignas | DeFi @DefiIgnas]

1/ How to get rich in crypto without getting lucky.

[The Ocean Cleanup @TheOceanCleanup]

According to our projections, we expect to be able to clean the entire Great Pacific Garbage Patch (containing 100,000,000kg of plastic) using ten systems based on System 03.

[holding it to pieces @_jafon_]

Oh my, what a brave and selfless act. I can’t believe that Yvon & family jumped off the capitalism ladder and gave Patagonia away to fight for the environment and climate change. Bravo! https://t.co/bzqkc4dQcE

[𝓜𝓪𝓽𝓽 𝓕𝓾𝓻𝓲𝓮 @Matt_Furie]

☮️

[Michael Paulauski @mike10010100]

Aaahahahahahaa holy shit the Dems actually stuck an amendment to the Clean Air Act into Biden's IRA that explicitly defines CO2 as an air pollutant, giving the EPA back its ability to regulate greenhouse gas emissions and directly addressing the Supreme Court's ruling.o

[holding it to pieces @_jafon_]

This was on a boat ride from Alaska to Washington

[olimpio @OlimpioCrypto]

An Ethereum Hard Fork is coming. You might want to:

• Maximize on-chain ETH for when it happens (withdraw from CEXes)
• Take an ETH loan
• Dump all PoW assets for ETHPoW
• Have a ready-to-go account on the greedy exchanges supporting the sh*t-chain

Thread on ETHPoW 🧙‍♂️👇

[Noah Smith 🐇🇺🇸🇺🇦🇹🇼 @Noahpinion]

The reason people on the Right say "the Left can't meme" is that the Left, like most human beings, uses memes for humor, while the online Right uses memes for tribal signaling.

[holding it to pieces @_jafon_]

She’s just the best. https://t.co/wnC4FxynlZ