MixedRealityToolkit-Unity | Mixed Reality Toolkit provides a set of components | Augmented Reality library

I am starting a project where I need the following two things:

1. raw sensor data from the HoloLens 2 to create a point cloud
2. functionality and ui from MRTK

The thing is that I'm using MRTK in a Unity project and the programming language is C#.
To get the depth information from the sensor I need to use the research mode, which is given by using HoloLens2ForCV and is written in C++.

I am currently still researching and trying to understand what direction I will take. What I have found so far is the following plugin for Unity, which is based on HoloLens2ForCV. Regardless of that, I would like to know what other possibilities there are and if anyone has already tried the same.

My Setup:

• Unity 2020.3
• MRTK 2.7.3
• latest state of HoloLens2ForCV

I started a project on Unity version 2019.4.24f1, imported MRTK 2.6.1 foundation, tools, etc and set everything up according to this guide:

https://microsoft.github.io/MixedRealityToolkit-Unity/Documentation/CrossPlatform/UsingARFoundation.html

The App starts and shows the elements that I placed in the testscene (buttons, etc), but the background of the app is black and there is no camera access question from my android system.

The device I am testing on is a Oneplus 3T and the minumum SDK requirements are met. How can I get this to work? I will post my MRTK and Player Settings below:

MRTK Settings:

Player Settings:

XR Settings: (Note that If I activate AR Core tick, it displays an error)

Error when AR Core is ticked:

Packages:

I want to implement a feature on HoloLens2 app, which allows user to draw/paint on specific surface in every part which was touched by the user.

So we have a flat plane and when I move my hand over this plane, spots below my hand should be coloured. How can I implement such feature keeping in mind, that HoloLens has limited processing power and calling Texture.Apply() every frame is unacceptable?

I tried to adopt a script from Eye tracking Heat map demo scene to use hand touch instead, but didn't success.

I've changed

I used the prefab HandMenu_Large_WorldLock_On_GrabAndPull as a basis for my hand menu. If grabbing the menu and placing it somewhere, I stays there and this behavior is fine for me. But now I want to walk away, looking again at my hand and the menu should reattach to my hand. How to set up HandConstraintPalmUp to achieve that?

What I tried:
I read here that I could set the SolverHandler to true or call the method HandConstraintPalmUp.StartWorldLockReattachCheckCoroutine() but this is actually not setting the SolverHandler to true, it just stays false. If running it via Holographic Emulation and setting the handler to true by clicking in the editor, it works, but not via OnFirstHandDetected or OnHandActivate of HandConstraintPamlUp. Also the documentation says:

When trying to set the hand constrained object to start following your hand again (based on whether it meets the activation criteria), set the SolverHandler's UpdateSolver to true.

But what is the criteria for the solver to set UpdateSolvers to true? What am I missing here?

Menu with default settings:
Why is for example the OnClick-Event on the button BtnClose working and my call on OnHandActiavte not?

I have tried to get hand mesh data from Hololens2 using MRTK V2 and Unity C#. Now, I can get hand mesh data with turning on Hand Mesh Visualization option and referring MRTK HandTracking guide.

Unfortunately, the visualization(drawing hand CG) is heavy workload. So, I would like to get hand mesh without turning on Hand Mesh Visualization option but OnHandMeshUpdated function is not called due to turning off Hand Mesh Visualization option.

Does anyone know how to get hand mesh data from Hololens2 without turning on Hand Mesh Visualization option?

Hand Mesh Visualization option

slight noob question and I may have missed something in the MRTK2-Unity docs/samples etc but I'm just getting back into Unity with the Hololens 2 and I'm looking for a simple example of dropping holograms onto the world mesh, rather like the original Hololens 1 MR 250 tutorial, using the old HoloToolKit: WorldAnchorManager and TapToPlace approach.

I'm sure this is quite a simple thing to achieve but can't seem to find an example now that the HoloToolKit "Manager Prefabs" approach has been replaced by the MRTK services etc.

I want to update some old HL1 projects to HL2 but the Porting Guide is either a bit unclear, or more likely, designed to be interpreted by clever people who know what they are doing...

cheers

i recently set up required software for development for the Microsoft HoloLens(1st. gen) with the newest version of the HoloLens(1st. gen) emulator (followed the MRTK-Guide: https://microsoft.github.io/MixedRealityToolkit-Unity/Documentation/GettingStartedWithTheMRTK.html#get-the-latest-mrtk-unity-packages). I already tested some example scenes from the MRTK on the emulator and everything works as expected. However, when i test the SpatialAwarenessMeshDemo, the app starts within the emulator with the unity start-screen. Then the scene is visible but after 1-2s the scene disappears and exceptions are thrown. Visual Studio Log: