Your photos, directed by Apple Intelligence — without ever leaving the iPhone.
Arraya AI turns up to 30 photos, videos and Live Photos into a cinematic reel of up to 60 seconds. Its generative animations — camera turns, dolly zooms, a gentle smile, moving skies — are painted by a Stable Diffusion model running on the phone itself. No photo is ever sent to an AI service.
Try the screens
A clickable demo of the main screens on iPhone 17 and the unfolded iPhone Duo, with all five themes in light and dark. Illustrations are drawn, not real user photos.
How it works
- Apple Intelligence writes captions and movie titles, and asks a question about each photo (“Which beach is this?”). Answers are remembered and suggested again when the same faces appear.
- Wide photos appear three to a screen with varied transitions; tall photos ride a rolling film strip; crops centre on faces in every format.
- Rendering continues with the phone locked and sends a notification when the movie is ready.
Five generative animations, all on-device
| Animation | What you see |
|---|---|
| X_depth | The camera turns 10–20° around the scene; what comes into view is painted to match. |
| X_smile | A gentle smile — repainted by the generative model at higher intensity. |
| X_background | People stay still while clouds drift, trees sway, water ripples and cars glide. |
| X_dolly in | The camera pushes in; people grow faster than the background. |
| X_dolly out | A dolly zoom: the subject stays locked while the background compresses. |
Under the hood
- Generative fill on the iPhone: Stable Diffusion 1.5 inpainting merged with LCM-LoRA (4 steps), 6-bit palettised, Neural Engine + GPU, about 850 MB. Apple's package can't do masked inpainting, so the loop was written from scratch in Swift.
- Scene understanding: Vision faces, landmarks, body pose and person masks; sky, greenery and water masks; MiDaS depth converted to Core ML.
- Clean plates and upscaling: generative fill first, LaMa as fallback, Real-ESRGAN for detail.
- Rendering: Core Image with custom Metal warp kernels, HEVC up to 4K, background processing tasks.
- Commerce: StoreKit 2 consumable top-up, tracked in the Keychain so purchases survive a reinstall.
Results so far
Problems we solved
- The “ghost”. A colour-inverted copy of a removed person appeared in X_depth. Root cause: high-contrast masks holding unclamped values like 2.5 and −1.5. A clamp after every contrast-built mask fixed it.
- A model that couldn't run on the phone. LivePortrait's 3D motion network failed on the iPhone's Neural Engine, so the effect moved to the Mac app and X_live moved to generative fill.
- The model painting new people. A rounded-box mask and a no-people prompt stopped it.
- Slow first preview. An early warm-up step with a progress bar driven by real loading milestones.
Stack
Status
Version 2 builds and runs on iPhone 17 and in the iPhone Duo simulator. Before launch: app logo, music library, the in-app purchase, licence notices, website and privacy policy. A Mac version, Arraya Desktop (up to 300 photos, 4K, 1–10 minute films), is fully coded.
Need something like this?
Tell us what your team does by hand today. We'll scope it into jobs and a price.