Mediapipe: "No registered object with name" with self-built apple_static_library

[lcycoding]

Dear mediapipe developers,

I'm currently trying to build my own .a library with the following script

apple_static_library(
    name = "MediapipeMainLibrary",
    platform_type = "ios",
    deps = [
        "//mediapipe/objc:mediapipe_framework_ios",
        "//mediapipe/graphs/object_detection:mobile_tflite_cpu",
        "@ios_opencv//:OpencvFramework",
    ],
)

mobile_tflite_cpu refers to my own calculator nodes

But when the library was imported to my own xcode project, it logged the following error

....
No registered object with name: TfLiteConverterCalculator; Unable to find Calculator "TfLiteConverterCalculator"
No registered object with name: TfLiteSSDInferenceWithNMSCalculator; Unable to find Calculator "TfLiteSSDInferenceWithNMSCalculator"
No registered object with name: DetectionLabelIdToTextCalculator; Unable to find Calculator "DetectionLabelIdToTextCalculator"
....

Basically, it could not recognize any nodes in my graph...
There's not much I know about the ios linker, but this code worked on my android project...
So there must be something wrong with ios build script.
Grateful for your comments!

[zjuturtle]

I have the same problem. I try to build it on CentOS with CMake

[lcycoding]

@zjuturtle
If you're building desktop version or android version.
Try with bazel, I think that will work just fine

[eknight7]

@lcycoding Are you able to use apple_static_library with an existing iOS example app?

[lcycoding]

@eknight7
Nope, it only works with include the library in ios_application

Directly put the library generated by apple_static_library into our own xcode project doesn't work :(

ios_application(
    name = "AlfredFaceDetectionCpuApp",
    bundle_id = "com.google.mediapipe.AlfredFaceDetectionCpu",
    families = [
        "iphone",
        "ipad",
    ],
    infoplists = ["Info.plist"],
    minimum_os_version = MIN_IOS_VERSION,
    provisioning_profile = "//mediapipe/examples/ios:provisioning_profile",
    deps = [
        ":ObjectDetectionCpuAppLibrary",
        "@ios_opencv//:OpencvFramework",
    ],
)

[zjuturtle]

I think I know why. It is the link problem.
Please note that mediapipe use "register" technique for most its component. You can check the code in framework/dep/registration.h. Actually it use dynamic initialization of global static variable to get its component register into FunctionRegistry class(I guess, I haven't read the code carefully). And according to C++ initialization order(detail here), this "register action" will happen before main() was called.
Please note, when you try to build a static library of mediapipe by hand(I use CMake for historical reasons), its all OK because you pack all code in .a file. You can use nm command to check your missing part (In your case it should be TfLiteConverterCalculator), you will find TfLiteConverterCalculator in .a library file. Then when you are linking this .a to your program, in cmake will like
~
add_executable(main main.cpp)
target_link_library(main mediapipe)
~
Default the link process will drop symbols not used in .a library. So if you check you program, the symbol is missing. Something like xxxCalculator is not linked into main.
How to fix? Some link flag like whole-archive in GCC will solve this problem.

[lcycoding]

@eknight7
I'm currently able to build usable .a file with some revised bazel rule, but I'll have to import opencv manually.

[GarrickLin]

@zjuturtle solved my problems.
It is worth mentioning that you should also remember to use -Wl,--no-whole-archive to ignore some unnecessary symbols. Or you will introduce other errors like "redefined symbol" or "undefined symbol".

In my case, I use this manner as following:

add_executable(hello_world hello_world.cc)
target_link_libraries(hello_world -Wl,--whole-archive core framework_stream_handler)
target_link_libraries(hello_world -Wl,--no-whole-archive ${framework_libs} ${calculators_libs} util)

[mapleyuan]

I think I know why. It is the link problem.
Please note that mediapipe use "register" technique for most its component. You can check the code in framework/dep/registration.h. Actually it use dynamic initialization of global static variable to get its component register into FunctionRegistry class(I guess, I haven't read the code carefully). And according to C++ initialization order(detail here), this "register action" will happen before main() was called.
Please note, when you try to build a static library of mediapipe by hand(I use CMake for historical reasons), its all OK because you pack all code in .a file. You can use nm command to check your missing part (In your case it should be TfLiteConverterCalculator), you will find TfLiteConverterCalculator in .a library file. Then when you are linking this .a to your program, in cmake will like

add_executable(main main.cpp)
target_link_library(main mediapipe)

Default the link process will drop symbols not used in .a library. So if you check you program, the symbol is missing. Something like xxxCalculator is not linked into main.
How to fix? Some link flag like whole-archive in GCC will solve this problem.

hi @zjuturtle , i still had the same problem. Could u give some advices?
I used the cmake and use the mediapipe at the subdirectory. like below:
ADD_SUBDIRECTORY(Detector)
link the target like this:
target_link_libraries(${PROJECT_NAME} libmediapipe.a -Wl,)

I got the same problem:

No registered object with name: FlowLimiterCalculator; Unable to find Calculator "FlowLimiterCalculator"
No registered object with name: GpuBufferToImageFrameCalculator; Unable to find Calculator "GpuBufferToImageFrameCalculator"
No registered object with name: ImagePropertiesCalculator; Unable to find Calculator "ImagePropertiesCalculator"

[kulich-ua]

@zjuturtle, thank you for your ideas! I think the reason of the issue, which you mentioned, is correct, but I was also not able to fix it by adjusting the compile process using different flags combinations. But I also have very little experience with bazel and compilers.



However I was able to build mediapipe hand tracking logic as a a static framework using the following workaround:



• I created dummy classes there I imported and referenced missing calculators like this

#import "MissingClasses.h"
#import <typeinfo>

#import "mediapipe/calculators/core/flow_limiter_calculator.h"

void MissingClasses::hello()
{
    typeid(::mediapipe::FlowLimiterCalculator);
}

• In case of FlowLimiterCalculator, I had to rename its file from flow_limiter_calculator.cc to flow_limiter_calculator.h and adjust mediapipe/calculators/core/BUILD appropriately

This approach worked for most of the missing classes, but for some others I had to apply slightly different workarounds.
So in case someone is interesting in this workaround and will have some other questions to this approach or need some more details I will be happy to help.

But I’m also aware that this approach is far from optimal and would also appreciate if someone shared a more straightforward solution.

[aaaron7]

Based on then idea come from @kulich-ua and @zjuturtle , I worked out a better workaround which can minimize the modification on original codebase.

• Modify REGISTER_CALCULATOR macro, add a function definition which simply make a reference to class. See the last line of following code snippet.

#define REGISTER_CALCULATOR(name)                                          \
  REGISTER_FACTORY_FUNCTION_QUALIFIED(::upipe::CalculatorBaseRegistry, \
                                      calculator_registration, name,       \
                                      absl::make_unique<name>);            \
  REGISTER_FACTORY_FUNCTION_QUALIFIED(                                     \
      ::upipe::internal::StaticAccessToCalculatorBaseRegistry,         \
      access_registration, name,                                           \
      absl::make_unique<                                                   \
          ::upipe::internal::StaticAccessToCalculatorBaseTyped<name>>);     \
  void register_##name(){typeid(name);};

• In your library interface implementation file. For example, MppLib.cc. Declare register functions of calculators you needed by extern, then make the call in your any of function that referred by the host app.

extern void register_FlowPackagerCalculator();
extern void register_ImageTransformationCalculator();
extern void register_MotionAnalysisCalculator();
//...
MppLib::MppLib(){
  register_FlowPackagerCalculator();
  register_ImageTransformationCalculator();
  register_MotionAnalysisCalculator();
//...
}

• 【Optional】Since the code in step2 is highly structured, we could do it automatically by python and bazel. Firstly we can use following command to extract all calculators that your target depends :

bazel cquery --config=ios_arm64 'deps(YOUR_TARGET_LABEL)' --noimplicit_deps --output label | grep 'calculator '

After get the calculator name list, we could generate the register code by few lines of python and inserted to our implementation files.