#361 · Primary category: Computer Vision

CoreML-in-ARKit

arkit augmented-reality coreml deep-learning ios machine-learning

Simple project to detect objects and display 3D labels above them in AR. This serves as a basic Template for an ARKit project to use CoreML.

Project last updated:08/23/21

GitHub Stars

1.7K

Forks

221

Contributors

2

License

MIT

Why we included this project

Building on-device object recognition into an iOS AR app usually means stitching together Vision, CoreML, and ARKit yourself. This sample shows how the three fit together: it feeds the live AR frame through the Inception V3 model and drops a SceneKit 3D text label at the detected object's location. The whole flow, from camera capture to label placement, lives in one view controller, so it is easy to read top to bottom and adapt. It is also honest about the rough edges, noting that SceneKit text can get polygon-heavy enough to crash on busy scenes and that the pixel format handed to the model (RGB versus YUV) is not fully settled. If you want a quick, readable starting point for prototyping AR object detection rather than a production framework, this.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category