Srt: Crash when buliding OBS Studio with ffmpeg linked against libsrt

[ocrete]

I rebuilt the OBS studio flatpak using the flathub project from https://github.com/flathub/com.obsproject.Studio and adding srt to the build and building ffmpeg with "--enable-libsrt", then when I try to run it, I get a crash with the following backtrace with git master of srt as of today:

```

0 std::operator+, std::allocator > (__lhs=0x7ffff4785099 ": ", __rhs=...)

at /usr/include/c++/8.2.0/bits/basic_string.tcc:1164

1 0x00007ffff4765cca in logging::Logger::Logger (this=0x7ffff604ab10 , functional_area=1, config=..., globprefix=...)

at /usr/include/c++/8.2.0/bits/basic_string.h:390

2 0x00007ffff4738fb4 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535)

at /usr/include/c++/8.2.0/ext/new_allocator.h:79

3 _GLOBAL__sub_I_core.cpp(void) () at /run/build/libsrt/srtcore/core.cpp:8704

``

[ocrete]

Here is the flatpak-builder manifest file I used to build it:
com.obsproject.Studio.json.txt

[ocrete]

FYI, I also tried against SRT v1.3.1

[ethouris]

Well, it may be controversial to call these things at the time of global initialization, but AFAIR the std::string type should not have any global initialization on itself (unlike, for example, iostream).

This starts from here:
logging::Logger blog(SRT_LOGFA_BSTATS, srt_logger_config, "SRT.b");

which then passes "SRT.b" string down to this Logger::Logger (assigned to globprefix):

m_prefix( globprefix == "" ? globprefix : ": " + globprefix)

What is FOA weird here is that this is the second object initialized this way, after glog, and for that there's no complaint.

I could try to replace it with using a static buffer and creating a string out of a glued-in array, but I'm not sure if this can help much because in the end it will still require creating an std::string object. If the simple operator+ crashes (while it should be language-guaranteed to work), it's hard to say if just a simple constructor wouldn't.

Note that this is a global initialization, the code is executed before main(), so there's not even any race condition in question.

[Ionic]

Experiencing the same issue.

As far as I've seen, srt is compiled in both C++11 and the default mode, ffmpeg is written in C, while OBS uses a mixture of C, C++11 and the GNU++11 dialect. The dialects should be fully compatible. Mixing C and C++ code also shouldn't cause trouble.

What feels odd is that for instance core.cpp is not compiled using any -std=... parameter, while a small subset of other files are compiled with -std=c++11. The default mode for GCC 7 and up is GNU++14. No idea if mixing C++11 and C++14 ABIs is prone to cause trouble.

I put together a very, very, very minimal test case and that did work correctly. Hardly surprising, since I essentially only put the globally initialized objects into one file (including the dependent classes) and compiled that using the flags srt used when compiling. That worked, but fails to replicate the more complex situation when dealing with obs anyway (i.e., loading shared libraries in a different context etc.) so really doesn't replicate anything useful.

[Ionic]

Force-appending -std=c++11 didn't change anything, so there goes that.

I did however notice something interesting.

Easy test cases are great, so I played around with something really minimal:

#include <cstdlib>

int main (int argc, char **argv) {
  return (EXIT_SUCCESS);
}

Standard flags didn't change the behavior, but (over-)linking in libobs immediately reproduced the crashes in libsrt's initialization.

On a hunch, I tried linking in libsrt first by explicitly specifying it before libobs. Program didn't crash.

Hence, the loading order seems to be important. This can also be exemplified by only linking in libobs, which would pull libsrt in and crash - but not if libsrt is preloaded at the very beginning via LD_PRELOAD.

I'm not sure what to make out of that. Most importantly, I still have no idea whether this is a bug in libsrt, obs or even the compiler.

[ethouris]

Mind that libsrt simply has its global initialization part, so it must be somehow ensured that the initialization is done before even initialization of anything else. That's the treat of C++, unfortunately, and the linker is likely unaware of that, especially if simultaneously another library (libobs possibly) has also its global constructor and it uses something that is dependent on a finished initialization of SRT.

Might be that SRT needs also some rework so that it relies less on the global C++ constructor (so that other libraries are free to make this mistake :D). We are kinda short of human resources to do it quickly, alas.

[boxerab]

Perhaps moving the global loggers into static variables would initialize them earlier and fix this problem ?

[boxerab]

Yeah, that fixes it. I will submit a pull request

[boxerab]

https://github.com/Haivision/srt/pull/567

[maxsharabayko]

Hi @boxerab,
Can you tell who is trying to access the logger before it is actually initialized? Can you do a debug build and check where the crash happens?

[ethouris]

One thing I don't understand here.

If the crash doesn't happen when loggers are made static, it means that their initialization is enforced to happen before libobs. So this has fixed it similar way as the forceful moving the libsrt in the linker order before libobs.

This means that the crash occurs in case when libobs is initialized first, then libsrt is initialized, still from the global constructor. However, if there's no crash when libobs is initialized second, then either the code doesn't use std::string anymore, or in the initialization chain there is made something that "fixes" the unstable state introduced by libobs.

It means that there still can be a problem with libobs initialization. Nevertheless, I'll try to prepare a fix by making the logging system not use std::string during the initialization time in the meantime.

[ethouris]

Ok, I have a trial fix on my repo replica with dev-fix-obs-crash. This changes the use of std::string for prefix handling into static char arrays. The prefix given by the logger has this time a limitation for up to 32 characters (at worst it's !!FATAL!!:SRT.c together, so it doesn't hurt). If you can please test if this version fixes the crash.

[boxerab]

Thanks, guys. By the way, here is a bit more background:
OBS doesn't link to libsrt. It links to libavcodec shared library, and libavcodec links
to libsrt shared library. So, so far, I have not touched any code other than libsrt.

Also , these loggers are singletons, so why not use the singleton pattern ? This way, they are not even
initialized if they aren't used.

[boxerab]

Here is the crash stack trace:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff3ce6e64 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) #1  0x00005555556fcbee in std::operator+<char, std::char_traits<char>, std::allocator<char> > (__lhs=0x7fffef7177c7 ": ", __rhs="SRT.b") at /usr/include/c++/7/bits/basic_string.tcc:1164
1164          __string_type __str;
#2  0x00007fffef6e1c80 in logging::Logger::Logger (this=0x7ffff4e4ebc1 <blog>, functional_area=1, config=..., globprefix="") at /home/aaron/src/srt/srtcore/logging.h:369
369             Fatal ( m_fa, LogLevel::fatal, "!!FATAL!!" + m_prefix, m_config )
#1  0x00005555556fcbee in std::operator+<char, std::char_traits<char>, std::allocator<char> > (__lhs=0x7fffef7177c7 ": ", __rhs="SRT.b") at /usr/include/c++/7/bits/basic_string.tcc:1164
1164          __string_type __str;
#2  0x00007fffef6e1c80 in logging::Logger::Logger (this=0x7ffff4e4ebc1 <blog>, functional_area=1, config=..., globprefix="") at /home/aaron/src/srt/srtcore/logging.h:369
369             Fatal ( m_fa, LogLevel::fatal, "!!FATAL!!" + m_prefix, m_config )
#0  0x00007ffff3ce6e64 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6

[ethouris]

Singletons in C++ (at least up to C++11) can't be used in a multithreaded environment (unless you want to put a heavy burden on performance and allow mutex-locking at every single access to the singleton object).

We were considering moving it to the srt_startup section. But as the problem here is when the OBS is initialized first, then SRT, then making them singletons will make this order happen always. The fix you mentioned above shifts the logger initialization to happen before OBS.

[boxerab]

@ethouris for loggers where they are only created once when first used, and then are thread safe from then on, the singleton will work. No need for mutex, because once logger is created, all operations are presumably thread safe. If not thread safe, then there is a bigger issue :) Also, runtime will do proper cleanup of static variable.

[ethouris]

@boxerab : Please test the branch dev-fix-obs-crash from my repo replica.

The last test definitely doesn't come from this branch because Logger::Logger now doesn't use std::string at all.

[boxerab]

@ethouris will do

[boxerab]

Unfortunately, @ethouris still crashes with your fix.

[boxerab]

```
Program received signal SIGSEGV, Segmentation fault.
0x00007fffef6e2990 in logging::Logger::Logger (this=0x7ffff4e4ebc1 , functional_area=1, config=..., logger_pfx=0x7fffef71d1d4 "SRT.b") at /home/aaron/src/srt/srtcore/logging.h:382
382 Fatal ( m_fa, LogLevel::fatal, "!!FATAL!!", logger_pfx, m_config )
Quit

[boxerab]

Can you tell who is trying to access the logger before it is actually initialized? Can you do a debug build and check where the crash happens?

@maxlovic it looks like the crash happens during initialization

[maxsharabayko]

@boxerab

Here is the crash stack trace:

2 0x00007fffef6e1c80 in logging::Logger::Logger (this=0x7ffff4e4ebc1 , functional_area=1, config=..., globprefix="") at /home/aaron/src/srt/srtcore/logging.h:369

369 Fatal ( m_fa, LogLevel::fatal, "!!FATAL!!" + m_prefix, m_config )

So, this is the place: "!!FATAL!!" + m_prefix (link to code).
In the constructor of class Logger, that calls
Fatal ( m_fa, LogLevel::fatal, "!!FATAL!!" + m_prefix, m_config )
m_prefix is initialized first, so should be no problem.
Interesting that other LogDispatchers (Debug, Note, etc.) are initialized, but Fatal dispatcher results in segmentation fault.

Do you have a hint on what type of crash is it? E.G. access violation reading location, allocation exception, etc.

Update
The program tries to allocate space for ": "+"SRT.b" (__lhs=0x7fffef7177c7 ": ", __rhs="SRT.b") and fails...

[boxerab]

@maxlovic from disassembly, it crashes in the movq instruction below:

                  _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC2Ev:
00007ffff3ce6e60:   lea     0x10(%rdi),%rax
00007ffff3ce6e64:   movq    $0x0,0x8(%rdi)
00007ffff3ce6e6c:   mov     %rax,(%rdi)
00007ffff3ce6e6f:   movb    $0x0,0x10(%rdi)
00007ffff3ce6e73:   retq    
00007ffff3ce6e74:   nopw    %cs:0x0(%rax,%rax,1)
00007ffff3ce6e7e:   xchg    %ax,%ax

The other LogDispatchers are probably corrupt as well; depends on the memory layout
where the crash will occur.

[boxerab]

By the way, there is a nice header-only logging project for C++

https://github.com/gabime/spdlog

Let someone else worry about this sort of issue ?

[maxsharabayko]

@boxerab,
Can you also show the function implementation in your /usr/include/c++/7/bits/basic_string.tcc:1164?

[boxerab]

@maxlovic here it is:

template<typename _CharT, typename _Traits, typename _Alloc>
    basic_string<_CharT, _Traits, _Alloc>
    operator+(const _CharT* __lhs,
          const basic_string<_CharT, _Traits, _Alloc>& __rhs)
    {
      __glibcxx_requires_string(__lhs);
      typedef basic_string<_CharT, _Traits, _Alloc> __string_type;
      typedef typename __string_type::size_type   __size_type;
      const __size_type __len = _Traits::length(__lhs);
      __string_type __str;
      __str.reserve(__len + __rhs.size());
      __str.append(__lhs, __len);
      __str.append(__rhs);
      return __str;
    }

First line is 1155, so line 1164 is:
__string_type __str;

[ethouris]

Guys, I'm getting more and more confused.

The fix on the branch I provided - dev-fix-obs-crash - does NOT use std::string during initialization time, and this is the only thing in question now. @maxlovic , please throw away std::string as a false track.

@boxerab , could you please get the exact backtrace for this crash in the version I fixed? I can see SIGSEGV, but in the instruction it shows i can't see anything that would make an illegal memory access. Would you be able to at least display the contents of all used objects?

[ethouris]

I have one more idea you can try out. I know it sounds stupid, but with unusual problems even stupid ideas may help.

Please try to comment out the 'blog' object definition in core.cpp. Occasionally, this FA somehow isn't used in the logging at all, so maybe the compiler or linker do something stupid about it. You may also want to comment out the extern declaration of it in core.h.

[boxerab]

@ethouris I think it would be helpful if you could repro this on your system.

Here is how I built my version of OBS

1. Build ffmpeg with libsrt:
   https://stackoverflow.com/questions/50967706/how-to-compile-ffmpeg-with-enabling-libsrt

2. Build OBS
   https://github.com/obsproject/obs-studio/wiki/Install-Instructions#linux-build-directions

3. Run OBS and it will crash immediately

[Ionic]

Full backtrace from the dev-fix-obs-crash branch:

#0  logging::Logger::Logger (this=0x7ffff6216f30 <blog>, functional_area=1, config=..., logger_pfx=0x7ffff1464a82 "SRT.b") at /var/tmp/paludis/net-libs-srt-9999/work/srt-9999/srtcore/logging.h:382
No locals.
#1  0x00007ffff140d963 in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /var/tmp/paludis/net-libs-srt-9999/work/srt-9999/srtcore/core.cpp:103
No locals.
#2  _GLOBAL__sub_I_core.cpp(void) () at /var/tmp/paludis/net-libs-srt-9999/work/srt-9999/srtcore/core.cpp:8704
No locals.
#3  0x00007ffff7fe422a in call_init (l=<optimized out>, argc=argc@entry=1, argv=argv@entry=0x7fffffffd1f8, env=env@entry=0x7fffffffd208) at dl-init.c:72
        j = <optimized out>
        jm = <optimized out>
        addrs = <optimized out>
        init_array = <optimized out>
#4  0x00007ffff7fe4336 in call_init (env=0x7fffffffd208, argv=0x7fffffffd1f8, argc=1, l=<optimized out>) at dl-init.c:118
        init_array = <optimized out>
        init_array = <optimized out>
        j = <optimized out>
        jm = <optimized out>
        addrs = <optimized out>
#5  _dl_init (main_map=0x7ffff7ffe130, argc=1, argv=0x7fffffffd1f8, env=0x7fffffffd208) at dl-init.c:119
        preinit_array = <optimized out>
        preinit_array_size = <optimized out>
        i = 34
#6  0x00007ffff7fd624a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
        library_path = 0x0
        version_info = 0
        any_debug = 0
        _dl_rtld_libname = {name = 0x5555555542a8 "/lib64/ld-linux-x86-64.so.2", next = 0x7ffff7ffdfa0 <newname>, dont_free = 0}
        relocate_time = 1248822662
        _dl_rtld_libname2 = {name = 0x0, next = 0x0, dont_free = 0}
        start_time = 13724032456652693
        tls_init_tp_called = true
        load_time = 289137344
        audit_list = 0x0
        preloadlist = 0x0
        __GI__dl_argv = 0x7fffffffd1f8
        _dl_argc = 1
        audit_list_string = 0x0
        _rtld_global = {_dl_ns = {{_ns_loaded = 0x7ffff7ffe130, _ns_nloaded = 211, _ns_main_searchlist = 0x7ffff7ffe3f0, _ns_global_scope_alloc = 0, _ns_unique_sym_table = {lock = {mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, 
                      __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, entries = 0x7fffe978a000, size = 251, n_elements = 112, free = 0x7ffff7fedc60 <free>}, 
              _ns_debug = {r_version = 0, r_map = 0x0, r_brk = 0, r_state = RT_CONSISTENT, r_ldbase = 0}}, {_ns_loaded = 0x0, _ns_nloaded = 0, _ns_main_searchlist = 0x0, _ns_global_scope_alloc = 0, _ns_unique_sym_table = {lock = {mutex = {__data = {__lock = 0, 
                      __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}}, entries = 0x0, size = 0, n_elements = 0, free = 0x0}, _ns_debug = {
                r_version = 0, r_map = 0x0, r_brk = 0, r_state = RT_CONSISTENT, r_ldbase = 0}} <repeats 15 times>}, _dl_nns = 1, _dl_load_lock = {mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __elision = 0, __list = {
                  __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, _dl_load_write_lock = {mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __elision = 0, 
                __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}}, _dl_load_adds = 211, _dl_initfirst = 0x0, _dl_cpuclock_offset = 13724032456678821, _dl_profile_map = 0x0, 
          _dl_num_relocations = 24555, _dl_num_cache_relocations = 11975, _dl_all_dirs = 0x7fffed326f20, _dl_rtld_map = {l_addr = 140737353961472, l_name = 0x5555555542a8 "/lib64/ld-linux-x86-64.so.2", l_ld = 0x7ffff7ffce18, l_next = 0x7ffff5480fd0, 
            l_prev = 0x7ffff5480a00, l_real = 0x7ffff7ffd990 <_rtld_local+2448>, l_ns = 0, l_libname = 0x7ffff7ffdff0 <_dl_rtld_libname>, l_info = {0x0, 0x0, 0x7ffff7ffce88, 0x7ffff7ffce78, 0x0, 0x7ffff7ffce38, 0x7ffff7ffce48, 0x7ffff7ffceb8, 0x7ffff7ffcec8, 
              0x7ffff7ffced8, 0x7ffff7ffce58, 0x7ffff7ffce68, 0x0, 0x0, 0x7ffff7ffce18, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ffff7ffce98, 0x0, 0x0, 0x7ffff7ffcea8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ffff7ffcf08, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ffff7ffcef8, 0x7ffff7ffcee8, 
              0x7ffff7ffcf18, 0x0, 0x7ffff7ffcf38, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ffff7ffcf28, 0x0 <repeats 25 times>, 0x7ffff7ffce28}, l_phdr = 0x7ffff7fd5040, l_entry = 0, l_phnum = 8, l_ldnum = 0, l_searchlist = {r_list = 0x0, r_nlist = 0}, 
            l_symbolic_searchlist = {r_list = 0x0, r_nlist = 0}, l_loader = 0x0, l_versions = 0x7fffe97a2800, l_nversions = 6, l_nbuckets = 58, l_gnu_bitmask_idxbits = 3, l_gnu_shift = 8, l_gnu_bitmask = 0x7ffff7fd5210, {l_gnu_buckets = 0x7ffff7fd5230, 
              l_chain = 0x7ffff7fd5230}, {l_gnu_chain_zero = 0x7ffff7fd5314, l_buckets = 0x7ffff7fd5314}, l_direct_opencount = 0, l_type = lt_library, l_relocated = 1, l_init_called = 1, l_global = 1, l_reserved = 0, l_phdr_allocated = 0, l_soname_added = 0, 
            l_faked = 0, l_need_tls_init = 0, l_auditing = 0, l_audit_any_plt = 0, l_removed = 0, l_contiguous = 0, l_symbolic_in_local_scope = 0, l_free_initfini = 0, l_cet = lc_unknown, l_rpath_dirs = {dirs = 0x0, malloced = 0}, l_reloc_result = 0x0, 
            l_versyms = 0x7ffff7fd58f4, l_origin = 0x0, l_map_start = 140737353961472, l_map_end = 140737354129712, l_text_end = 140737354086640, l_scope_mem = {0x0, 0x0, 0x0, 0x0}, l_scope_max = 0, l_scope = 0x0, l_local_scope = {0x0, 0x0}, l_file_id = {dev = 0, 
              ino = 0}, l_runpath_dirs = {dirs = 0x0, malloced = 0}, l_initfini = 0x0, l_reldeps = 0x0, l_reldepsmax = 0, l_used = 1, l_feature_1 = 0, l_flags_1 = 0, l_flags = 0, l_idx = 0, l_mach = {plt = 0, gotplt = 0, tlsdesc_table = 0x0}, l_lookup_cache = {
              sym = 0x7ffff7fd5418, type_class = 1, value = 0x7ffff5480a00, ret = 0x7ffff527c1c0}, l_tls_initimage = 0x0, l_tls_initimage_size = 0, l_tls_blocksize = 0, l_tls_align = 0, l_tls_firstbyte_offset = 0, l_tls_offset = 0, l_tls_modid = 0, 
            l_tls_dtor_count = 0, l_relro_addr = 161248, l_relro_size = 2592, l_serial = 0, l_audit = 0x7ffff7ffde08 <_rtld_local+3592>}, audit_data = {{cookie = 0, bindflags = 0} <repeats 16 times>}, _dl_rtld_lock_recursive = 0x7ffff544f1a0 <pthread_mutex_lock>, 
          _dl_rtld_unlock_recursive = 0x7ffff5450ac0 <pthread_mutex_unlock>, _dl_x86_feature_1 = {0, 0}, _dl_x86_legacy_bitmap = {0, 0}, _dl_make_stack_executable_hook = 0x7ffff544b640 <__make_stacks_executable>, _dl_stack_flags = 6, _dl_tls_dtv_gaps = false, 
          _dl_tls_max_dtv_idx = 12, _dl_tls_dtv_slotinfo_list = 0x7fffe9792a90, _dl_tls_static_nelem = 12, _dl_tls_static_size = 4864, _dl_tls_static_used = 888, _dl_tls_static_align = 64, _dl_initial_dtv = 0x7fffe9790360, _dl_tls_generation = 1, 
          _dl_init_static_tls = 0x7ffff544be30 <__pthread_init_static_tls>, _dl_wait_lookup_done = 0x7ffff544bf80 <__wait_lookup_done>, _dl_scope_free_list = 0x0}
--Type <RET> for more, q to quit, c to continue without paging--
        _rtld_global_ro = {_dl_debug_mask = 0, _dl_osversion = 267264, _dl_platform = 0x7ffff7ff5c8c "haswell", _dl_platformlen = 7, _dl_pagesize = 4096, _dl_inhibit_cache = 0, _dl_initial_searchlist = {r_list = 0x7fffe97b1908, r_nlist = 210}, _dl_clktck = 100, 
          _dl_verbose = 0, _dl_debug_fd = 2, _dl_lazy = 1, _dl_bind_not = 0, _dl_dynamic_weak = 0, _dl_fpu_control = 895, _dl_correct_cache_id = 771, _dl_hwcap = 2, _dl_auxv = 0x7fffffffd5e0, _dl_x86_cpu_features = {kind = arch_kind_intel, max_cpuid = 22, cpuid = {{
                eax = 591593, ebx = 17827840, ecx = 2147154879, edx = 3219913727}, {eax = 0, ebx = 43806655, ecx = 0, edx = 2617245696}, {eax = 0, ebx = 0, ecx = 289, edx = 739248128}}, family = 6, model = 158, xsave_state_size = 960, xsave_state_full_size = 1152, 
            feature = {5557489}, data_cache_size = 0, shared_cache_size = 0, non_temporal_threshold = 0}, _dl_x86_hwcap_flags = {"sse2\000\000\000\000", "x86_64\000\000", "avx512_1"}, _dl_x86_platforms = {"i586\000\000\000\000", "i686\000\000\000\000", "haswell\000", 
            "xeon_phi"}, _dl_inhibit_rpath = 0x0, _dl_origin_path = 0x0, _dl_use_load_bias = 0, _dl_profile = 0x0, _dl_profile_output = 0x7ffff7ff407c "/var/tmp", _dl_trace_prelink = 0x0, _dl_trace_prelink_map = 0x0, _dl_init_all_dirs = 0x7fffed326f20, 
          _dl_sysinfo_dso = 0x7ffff7fd3000, _dl_sysinfo_map = 0x7ffff7ffe6d0, _dl_hwcap2 = 0, _dl_debug_printf = 0x7ffff7fe4f10 <_dl_debug_printf>, _dl_mcount = 0x7ffff7fe6300 <__GI__dl_mcount>, _dl_lookup_symbol_x = 0x7ffff7fdf0c0 <_dl_lookup_symbol_x>, 
          _dl_open = 0x7ffff7fe79b0 <_dl_open>, _dl_close = 0x7ffff7fe96b0 <_dl_close>, _dl_tls_get_addr_soft = 0x7ffff7fe7270 <_dl_tls_get_addr_soft>, _dl_discover_osversion = 0x7ffff7fed850 <_dl_discover_osversion>, _dl_audit = 0x0, _dl_naudit = 0}
        _dl_skip_args = 0
        __pointer_chk_guard_local = 11590407887763127805
#7  0x0000000000000001 in ?? ()
No symbol table info available.
#8  0x00007fffffffd743 in ?? ()
No symbol table info available.
#9  0x0000000000000000 in ?? ()
No symbol table info available.

No std::string anymore, but still crashing... in the constructor? For no apparent reason? The whole issue might also just as well stem from libobs corrupting the heap or who knows what. I've tried executing obs with valgrind/memcheck, but the output is... huge.

Even more weirdly, force-loading libsrt earlie(r|st) doesn't help obs a lot, since it will still crash - unlike a simple "Hello World"-like program that does nothing but linking libsrt and libobs. Guess when?

# LD_PRELOAD=/usr/lib64/libsrt.so gdb obs
[...]
(gdb) bt full
#0  0x00007ffff7fcd040 in blog () from /usr/lib64/libsrt.so
No symbol table info available.
#1  0x00007ffff61a7e9e in dbus_sleep_info_create () at /var/tmp/paludis/media-video-obs-studio-22.0.3/work/obs-studio-22.0.3/libobs/util/platform-nix-dbus.c:107
        service = 0x7ffff6212c08 <services+72>
        i = 3
        info = <optimized out>
        err = {name = 0x0, message = 0x0, dummy1 = 1, dummy2 = 0, dummy3 = 0, dummy4 = 0, dummy5 = 0, padding1 = 0x7fffffffc670}
#2  0x00007ffff61a7839 in os_inhibit_sleep_create (reason=0x55555570411f "OBS Video/audio") at /var/tmp/paludis/media-video-obs-studio-22.0.3/work/obs-studio-22.0.3/libobs/util/platform-nix.c:537
        info = <optimized out>
        set = {__val = {0, 140737312692468, 140737316641264, 2, 140737314858224, 1, 140737321930056, 140737488341088, 140737314858224, 93824994463648, 140737488340784, 140737317561343, 140737488341088, 2648596224, 140737350086880, 14052779085560438528}}
#3  0x00005555555fc52f in OBSApp::OBSApp (this=0x7fffffffc820, argc=<optimized out>, argv=<optimized out>, store=<optimized out>) at /var/tmp/paludis/media-video-obs-studio-22.0.3/work/obs-studio-22.0.3/UI/obs-app.cpp:1032
No locals.
#4  0x00005555555e4eed in main (argc=1, argv=0x7fffffffd1d8) at /var/tmp/paludis/media-video-obs-studio-22.0.3/work/obs-studio-22.0.3/UI/obs-app.cpp:1651
        logFile = {<std::basic_iostream<char, std::char_traits<char> >> = {<std::basic_istream<char, std::char_traits<char> >> = {<std::basic_ios<char, std::char_traits<char> >> = {<std::ios_base> = {
                  _vptr.ios_base = 0x7ffff5778400 <vtable for std::basic_fstream<char, std::char_traits<char> >+104>, static boolalpha = std::_S_boolalpha, static dec = std::_S_dec, static fixed = std::_S_fixed, static hex = std::_S_hex, 
                  static internal = std::_S_internal, static left = std::_S_left, static oct = std::_S_oct, static right = std::_S_right, static scientific = std::_S_scientific, static showbase = std::_S_showbase, static showpoint = std::_S_showpoint, 
                  static showpos = std::_S_showpos, static skipws = std::_S_skipws, static unitbuf = std::_S_unitbuf, static uppercase = std::_S_uppercase, static adjustfield = std::_S_adjustfield, static basefield = std::_S_basefield, 
                  static floatfield = std::_S_floatfield, static badbit = std::_S_badbit, static eofbit = std::_S_eofbit, static failbit = std::_S_failbit, static goodbit = std::_S_goodbit, static app = std::_S_app, static ate = std::_S_ate, 
                  static binary = std::_S_bin, static in = std::_S_in, static out = std::_S_out, static trunc = std::_S_trunc, static beg = std::_S_beg, static cur = std::_S_cur, static end = std::_S_end, _M_precision = 6, _M_width = 0, _M_flags = 4098, 
                  _M_exception = std::_S_goodbit, _M_streambuf_state = std::_S_goodbit, _M_callbacks = 0x0, _M_word_zero = {_M_pword = 0x0, _M_iword = 0}, _M_local_word = {{_M_pword = 0x0, _M_iword = 0}, {_M_pword = 0x0, _M_iword = 0}, {_M_pword = 0x0, _M_iword = 0}, {
                      _M_pword = 0x0, _M_iword = 0}, {_M_pword = 0x0, _M_iword = 0}, {_M_pword = 0x0, _M_iword = 0}, {_M_pword = 0x0, _M_iword = 0}, {_M_pword = 0x0, _M_iword = 0}}, _M_word_size = 8, _M_word = 0x7fffffffcbe8, _M_ios_locale = {static none = 0, 
                    static ctype = 1, static numeric = 2, static collate = 4, static time = 8, static monetary = 16, static messages = 32, static all = 63, _M_impl = 0x7ffff57832a0 <(anonymous namespace)::c_locale_impl>, 
                    static _S_classic = 0x7ffff57832a0 <(anonymous namespace)::c_locale_impl>, static _S_global = 0x7ffff57832a0 <(anonymous namespace)::c_locale_impl>, static _S_categories = 0x7ffff5771b80 <__gnu_cxx::category_names>, static _S_once = 2, 
                    static _S_twinned_facets = 0x7ffff57734c0 <std::locale::_S_twinned_facets>}}, _M_tie = 0x0, _M_fill = 0 '\000', _M_fill_init = false, _M_streambuf = 0x7fffffffcab8, _M_ctype = 0x7ffff5782cc0 <(anonymous namespace)::ctype_c>, 
                _M_num_put = 0x7ffff5782c50 <(anonymous namespace)::num_put_c>, _M_num_get = 0x7ffff5782c60 <(anonymous namespace)::num_get_c>}, _vptr.basic_istream = 0x7ffff57783b0 <vtable for std::basic_fstream<char, std::char_traits<char> >+24>, 
              _M_gcount = 0}, <std::basic_ostream<char, std::char_traits<char> >> = {_vptr.basic_ostream = 0x7ffff57783d8 <vtable for std::basic_fstream<char, std::char_traits<char> >+64>}, <No data fields>}, 
          _M_filebuf = {<std::basic_streambuf<char, std::char_traits<char> >> = {_vptr.basic_streambuf = 0x7ffff5778040 <vtable for std::basic_filebuf<char, std::char_traits<char> >+16>, _M_in_beg = 0x0, _M_in_cur = 0x0, _M_in_end = 0x0, _M_out_beg = 0x0, 
              _M_out_cur = 0x0, _M_out_end = 0x0, _M_buf_locale = {static none = 0, static ctype = 1, static numeric = 2, static collate = 4, static time = 8, static monetary = 16, static messages = 32, static all = 63, 
                _M_impl = 0x7ffff57832a0 <(anonymous namespace)::c_locale_impl>, static _S_classic = 0x7ffff57832a0 <(anonymous namespace)::c_locale_impl>, static _S_global = 0x7ffff57832a0 <(anonymous namespace)::c_locale_impl>, 
                static _S_categories = 0x7ffff5771b80 <__gnu_cxx::category_names>, static _S_once = 2, static _S_twinned_facets = 0x7ffff57734c0 <std::locale::_S_twinned_facets>}}, _M_lock = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, 
                __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, _M_file = {_M_cfile = 0x0, _M_cfile_created = false}, _M_mode = 0, _M_state_beg = {__count = 0, __value = {__wch = 0, 
                __wchb = "\000\000\000"}}, _M_state_cur = {__count = 0, __value = {__wch = 0, __wchb = "\000\000\000"}}, _M_state_last = {__count = 0, __value = {__wch = 0, __wchb = "\000\000\000"}}, _M_buf = 0x0, _M_buf_size = 8192, _M_buf_allocated = false, 
            _M_reading = false, _M_writing = false, _M_pback = 0 '\000', _M_pback_cur_save = 0x0, _M_pback_end_save = 0x0, _M_pback_init = false, _M_codecvt = 0x7ffff5782c30 <(anonymous namespace)::codecvt_c>, _M_ext_buf = 0x0, _M_ext_buf_size = 0, _M_ext_next = 0x0, 
            _M_ext_end = 0x0}}
        ret = <optimized out>

Yeah, the first time using blog().

[boxerab]

Interesting. Does libsrt get tested with valgrind, btw ?

[boxerab]

So, I also ran valgrind on gstreamer sending an srt stream :

valgrind gst-launch-1.0 v4l2src ! video/x-raw, height=480, width=640 ! videoconvert ! x264enc tune=zerolatency ! video/x-h264, profile=high ! mpegtsmux ! srtserversink uri=srt://:7001/
==6846== Memcheck, a memory error detector
==6846== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==6846== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==6846== Command: gst-launch-1.0 v4l2src ! video/x-raw, height=480, width=640 ! videoconvert ! x264enc tune=zerolatency ! video/x-h264, profile=high ! mpegtsmux ! srtserversink uri=srt://:7001/
==6846== 
Setting pipeline to PAUSED ...
==6846== Syscall param msync(start) points to uninitialised byte(s)
==6846==    at 0x56F7BC9: msync (msync.c:25)
==6846==    by 0x5EFE2F3: ??? (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0x5F02230: ??? (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0x5F0253E: ??? (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0x5F02A98: ??? (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0x5EFEE70: _ULx86_64_step (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0x5EFD940: _Unwind_RaiseException (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xCC22C66: __cxa_throw (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==6846==    by 0xBDE3928: CChannel::open(sockaddr const*) (channel.cpp:145)
==6846==    by 0xBDC4930: CUDTUnited::updateMux(CUDTSocket*, sockaddr const*, int const*) (api.cpp:1700)
==6846==    by 0xBDBF7DA: CUDTUnited::bind(int, sockaddr const*, int) (api.cpp:591)
==6846==    by 0xBDC595D: CUDT::bind(int, sockaddr const*, int) (api.cpp:1937)
==6846==  Address 0x1ffeffe350 is on thread 1's stack
==6846== 
==6846== Invalid read of size 8
==6846==    at 0x5EFDEF6: _Ux86_64_setcontext (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xBDBF7DA: CUDTUnited::bind(int, sockaddr const*, int) (api.cpp:591)
==6846==    by 0xBDC595D: CUDT::bind(int, sockaddr const*, int) (api.cpp:1937)
==6846==    by 0xBE470ED: srt_bind (srt_c_api.cpp:52)
==6846==    by 0xB7B2DF2: gst_srt_object_wait_connect.isra.1 (gstsrtobject.c:729)
==6846==    by 0xB7B42BA: gst_srt_object_open_connection (gstsrtobject.c:914)
==6846==    by 0xB7B42BA: gst_srt_object_open_full (gstsrtobject.c:1007)
==6846==    by 0xB7B61B3: gst_srt_sink_start (gstsrtsink.c:150)
==6846==    by 0x94E9C1C: gst_base_sink_change_state (gstbasesink.c:5161)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x4E78701: gst_bin_element_set_state (gstbin.c:2601)
==6846==    by 0x4E78701: gst_bin_change_state_func (gstbin.c:2943)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==  Address 0x1ffeffe588 is on thread 1's stack
==6846==  2504 bytes below stack pointer
==6846== 
==6846== Invalid read of size 8
==6846==    at 0x5EFDEFE: _Ux86_64_setcontext (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xBDBF7DA: CUDTUnited::bind(int, sockaddr const*, int) (api.cpp:591)
==6846==    by 0xBDC595D: CUDT::bind(int, sockaddr const*, int) (api.cpp:1937)
==6846==    by 0xBE470ED: srt_bind (srt_c_api.cpp:52)
==6846==    by 0xB7B2DF2: gst_srt_object_wait_connect.isra.1 (gstsrtobject.c:729)
==6846==    by 0xB7B42BA: gst_srt_object_open_connection (gstsrtobject.c:914)
==6846==    by 0xB7B42BA: gst_srt_object_open_full (gstsrtobject.c:1007)
==6846==    by 0xB7B61B3: gst_srt_sink_start (gstsrtsink.c:150)
==6846==    by 0x94E9C1C: gst_base_sink_change_state (gstbasesink.c:5161)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x4E78701: gst_bin_element_set_state (gstbin.c:2601)
==6846==    by 0x4E78701: gst_bin_change_state_func (gstbin.c:2943)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==  Address 0x1ffeffe578 is on thread 1's stack
==6846==  2512 bytes below stack pointer
==6846== 
==6846== Invalid read of size 8
==6846==    at 0x5EFDF05: _Ux86_64_setcontext (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xBDBF7DA: CUDTUnited::bind(int, sockaddr const*, int) (api.cpp:591)
==6846==    by 0xBDC595D: CUDT::bind(int, sockaddr const*, int) (api.cpp:1937)
==6846==    by 0xBE470ED: srt_bind (srt_c_api.cpp:52)
==6846==    by 0xB7B2DF2: gst_srt_object_wait_connect.isra.1 (gstsrtobject.c:729)
==6846==    by 0xB7B42BA: gst_srt_object_open_connection (gstsrtobject.c:914)
==6846==    by 0xB7B42BA: gst_srt_object_open_full (gstsrtobject.c:1007)
==6846==    by 0xB7B61B3: gst_srt_sink_start (gstsrtsink.c:150)
==6846==    by 0x94E9C1C: gst_base_sink_change_state (gstbasesink.c:5161)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x4E78701: gst_bin_element_set_state (gstbin.c:2601)
==6846==    by 0x4E78701: gst_bin_change_state_func (gstbin.c:2943)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==  Address 0x1ffeffe548 is on thread 1's stack
==6846==  2560 bytes below stack pointer
==6846== 
==6846== Invalid read of size 8
==6846==    at 0x5EFDEF6: _Ux86_64_setcontext (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xBDC595D: CUDT::bind(int, sockaddr const*, int) (api.cpp:1937)
==6846==    by 0xBE470ED: srt_bind (srt_c_api.cpp:52)
==6846==    by 0xB7B2DF2: gst_srt_object_wait_connect.isra.1 (gstsrtobject.c:729)
==6846==    by 0xB7B42BA: gst_srt_object_open_connection (gstsrtobject.c:914)
==6846==    by 0xB7B42BA: gst_srt_object_open_full (gstsrtobject.c:1007)
==6846==    by 0xB7B61B3: gst_srt_sink_start (gstsrtsink.c:150)
==6846==    by 0x94E9C1C: gst_base_sink_change_state (gstbasesink.c:5161)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x4E78701: gst_bin_element_set_state (gstbin.c:2601)
==6846==    by 0x4E78701: gst_bin_change_state_func (gstbin.c:2943)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==  Address 0x1ffeffe7f8 is on thread 1's stack
==6846==  2552 bytes below stack pointer
==6846== 
==6846== Invalid read of size 8
==6846==    at 0x5EFDEFE: _Ux86_64_setcontext (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xBDC595D: CUDT::bind(int, sockaddr const*, int) (api.cpp:1937)
==6846==    by 0xBE470ED: srt_bind (srt_c_api.cpp:52)
==6846==    by 0xB7B2DF2: gst_srt_object_wait_connect.isra.1 (gstsrtobject.c:729)
==6846==    by 0xB7B42BA: gst_srt_object_open_connection (gstsrtobject.c:914)
==6846==    by 0xB7B42BA: gst_srt_object_open_full (gstsrtobject.c:1007)
==6846==    by 0xB7B61B3: gst_srt_sink_start (gstsrtsink.c:150)
==6846==    by 0x94E9C1C: gst_base_sink_change_state (gstbasesink.c:5161)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x4E78701: gst_bin_element_set_state (gstbin.c:2601)
==6846==    by 0x4E78701: gst_bin_change_state_func (gstbin.c:2943)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==  Address 0x1ffeffe7e8 is on thread 1's stack
==6846==  2560 bytes below stack pointer
==6846== 
==6846== Invalid read of size 8
==6846==    at 0x5EFDF05: _Ux86_64_setcontext (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xBDC595D: CUDT::bind(int, sockaddr const*, int) (api.cpp:1937)
==6846==    by 0xBE470ED: srt_bind (srt_c_api.cpp:52)
==6846==    by 0xB7B2DF2: gst_srt_object_wait_connect.isra.1 (gstsrtobject.c:729)
==6846==    by 0xB7B42BA: gst_srt_object_open_connection (gstsrtobject.c:914)
==6846==    by 0xB7B42BA: gst_srt_object_open_full (gstsrtobject.c:1007)
==6846==    by 0xB7B61B3: gst_srt_sink_start (gstsrtsink.c:150)
==6846==    by 0x94E9C1C: gst_base_sink_change_state (gstbasesink.c:5161)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x4E78701: gst_bin_element_set_state (gstbin.c:2601)
==6846==    by 0x4E78701: gst_bin_change_state_func (gstbin.c:2943)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==  Address 0x1ffeffe7b8 is on thread 1's stack
==6846==  2608 bytes below stack pointer
==6846== 
==6846== Invalid read of size 8
==6846==    at 0x5EFDEF6: _Ux86_64_setcontext (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xBE470ED: srt_bind (srt_c_api.cpp:52)
==6846==    by 0xB7B2DF2: gst_srt_object_wait_connect.isra.1 (gstsrtobject.c:729)
==6846==    by 0xB7B42BA: gst_srt_object_open_connection (gstsrtobject.c:914)
==6846==    by 0xB7B42BA: gst_srt_object_open_full (gstsrtobject.c:1007)
==6846==    by 0xB7B61B3: gst_srt_sink_start (gstsrtsink.c:150)
==6846==    by 0x94E9C1C: gst_base_sink_change_state (gstbasesink.c:5161)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x4E78701: gst_bin_element_set_state (gstbin.c:2601)
==6846==    by 0x4E78701: gst_bin_change_state_func (gstbin.c:2943)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x10BA08: main (gst-launch.c:1133)
==6846==  Address 0x1ffeffea98 is on thread 1's stack
==6846==  1992 bytes below stack pointer
==6846== 
==6846== Invalid read of size 8
==6846==    at 0x5EFDEFE: _Ux86_64_setcontext (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xBE470ED: srt_bind (srt_c_api.cpp:52)
==6846==    by 0xB7B2DF2: gst_srt_object_wait_connect.isra.1 (gstsrtobject.c:729)
==6846==    by 0xB7B42BA: gst_srt_object_open_connection (gstsrtobject.c:914)
==6846==    by 0xB7B42BA: gst_srt_object_open_full (gstsrtobject.c:1007)
==6846==    by 0xB7B61B3: gst_srt_sink_start (gstsrtsink.c:150)
==6846==    by 0x94E9C1C: gst_base_sink_change_state (gstbasesink.c:5161)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x4E78701: gst_bin_element_set_state (gstbin.c:2601)
==6846==    by 0x4E78701: gst_bin_change_state_func (gstbin.c:2943)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x10BA08: main (gst-launch.c:1133)
==6846==  Address 0x1ffeffea88 is on thread 1's stack
==6846==  2000 bytes below stack pointer
==6846== 
==6846== Invalid read of size 8
==6846==    at 0x5EFDF05: _Ux86_64_setcontext (in /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1)
==6846==    by 0xBE470ED: srt_bind (srt_c_api.cpp:52)
==6846==    by 0xB7B2DF2: gst_srt_object_wait_connect.isra.1 (gstsrtobject.c:729)
==6846==    by 0xB7B42BA: gst_srt_object_open_connection (gstsrtobject.c:914)
==6846==    by 0xB7B42BA: gst_srt_object_open_full (gstsrtobject.c:1007)
==6846==    by 0xB7B61B3: gst_srt_sink_start (gstsrtsink.c:150)
==6846==    by 0x94E9C1C: gst_base_sink_change_state (gstbasesink.c:5161)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x4E78701: gst_bin_element_set_state (gstbin.c:2601)
==6846==    by 0x4E78701: gst_bin_change_state_func (gstbin.c:2943)
==6846==    by 0x4E9B5DD: gst_element_change_state (gstelement.c:2952)
==6846==    by 0x4E9BD18: gst_element_set_state_func (gstelement.c:2906)
==6846==    by 0x10BA08: main (gst-launch.c:1133)
==6846==  Address 0x1ffeffea58 is on thread 1's stack
==6846==  2048 bytes below stack pointer
==6846== 
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstSRTSink:srtsink0: GStreamer error: state change failed and some element failed to post a proper error message with the reason for the failure.
Additional debug info:
gstbasesink.c(5368): gst_base_sink_change_state (): /GstPipeline:pipeline0/GstSRTSink:srtsink0:
Failed to start
Setting pipeline to NULL ...
Freeing pipeline ...
==6846== 
==6846== HEAP SUMMARY:
==6846==     in use at exit: 736,858 bytes in 3,409 blocks
==6846==   total heap usage: 50,785 allocs, 47,376 frees, 7,111,697 bytes allocated
==6846== 
==6846== LEAK SUMMARY:
==6846==    definitely lost: 16,600 bytes in 4 blocks
==6846==    indirectly lost: 116 bytes in 5 blocks
==6846==      possibly lost: 4,708 bytes in 61 blocks
==6846==    still reachable: 693,610 bytes in 3,174 blocks
==6846==                       of which reachable via heuristic:
==6846==                         length64           : 784 bytes in 16 blocks
==6846==                         newarray           : 1,664 bytes in 24 blocks
==6846==         suppressed: 0 bytes in 0 blocks
==6846== Rerun with --leak-check=full to see details of leaked memory
==6846== 
==6846== For counts of detected and suppressed errors, rerun with: -v
==6846== Use --track-origins=yes to see where uninitialised values come from
==6846== ERROR SUMMARY: 14 errors from 10 contexts (suppressed: 0 from 0)

[boxerab]

Nothing too bad in libsrt, so obs-studio may be the problem.

[boxerab]

What feels odd is that for instance core.cpp is not compiled using any -std=... parameter, while a small subset of other files are compiled with -std=c++11. The default mode for GCC 7 and up is GNU++14. No idea if mixing C++11 and C++14 ABIs is prone to cause trouble.

@Ionic so for my singleton code to be thread safe, core.cpp would need to be compiled with -std=c++11. Before c++11, local static initialization was not guaranteed to be thread safe.

[Ionic]

Please try to comment out the 'blog' object definition in core.cpp. Occasionally, this FA somehow isn't used in the logging at all, so maybe the compiler or linker do something stupid about it. You may also want to comment out the extern declaration of it in core.h.

Uh, weird, that seems to alleviate the crash. Tested on top of your dev-fix-obs-crash branch. I would have expected it to be crashing in mglog's constructor instead, but it doesn't.

So, I also ran valgrind on gstreamer sending an srt stream :

Well, my original idea was to start obs in valgrind to... maybe easily spot some heap corruption, since I did (and still do) believe that libobs is doing something funny. After being flooded with megabytes of output before it crashed, I gave up on that idea.

@Ionic so for my singleton code to be thread safe, core.cpp would need to be compiled with -std=c++11. Before c++11, local static initialization was not guaranteed to be thread safe.

That's already the case, IIRC. At least if C++11-support is not completely disabled (which should in turn only disable some parts of srt's distribution), -std=c++11 is added to the default set of CXXFLAGS. That said, you've just given another good reason against static objects - the core library should not need to depend upon C++11 features (at least for now), since there are a lot of systems out there with old compilers. Missing functionality due to a feature being disabled is better than the whole distribution being unusable in such a case.

[ethouris]

Just came to my mind, and I feel ashamed of myself that it came so late to my mind.

The blog name in the global namespace may be already used by some other library. That's actually the risk embracing every other global variable. So, this may help by different reason that I thought in the beginning. We'll definitely need to rename all global variables that we use.

[ethouris]

BTW. I'm trying to currently build and install flatpak and obs-studio from it. I have some problems. Would you be able to contact me more directly? (mmalecki at haivision.com, for example)

[boxerab]

@Ionic thanks, if this fixes it then let's get that merged.
@ethouris perhaps we could use a namespace to avoid name pollution from other projects ?

[ethouris]

Ok, I added a few other changes on the dev-fix-obs-crash branch:

1. The blog object is commented out. Its place and FA symbol can be taken for something else in future, but currently it's unused, better to keep it out.
2. All *log objects have been put into logging namespace. This should keep at least all current potential name conflicts away.

We probably still need to think of adding the srt namespace to whole code to minimize any future name conflicts, but it's a bigger task.

In the meantime @boxerab is helping me with reproducing it myself.

[boxerab]

@ethouris thanks, I will test this

[boxerab]

@ethouris still crashing

[boxerab]

@ethouris sorry, I didn't pull your latest changes. All working now. Thanks! LGTM

[ethouris]

Ok, now you may also try to run nm on the whole bundle after building and linking and see, which library provides the blog symbol. You can use an "nm on steroids", nma.tcl from my repo: gitlab.com/posix-tools/posix-tools.

Just to confirm that this was the root cause.

[Ionic]

Oh, crap, yes. I'll go figure it out.

[Ionic]

It seems that libobs itself defines a global blog symbol, so... yeah. It's not like any of its dependencies - at least on my system - would do something similar, even though the ffmpeg plugin naturally references it (I haven't rebuilt obs-studio or ffmpeg after commenting out the blog symbol in libsrt, so that makes sense).

[boxerab]

aha!! I checked the OBS source and there are 3968 references to blog. I guess blog for base log ?

[boxerab]

I guess global variables really are evil

[boxerab]

:)

[Ionic]

Yeah, probably.

So, that explains that. How about removing the commit that changes std::string usage to fixed-length char arrays, since that doesn't seem to be a problem after all and re-enabling the blog logging symbol, since it now should be in the srt_logging namespace?

[boxerab]

Also, is the plan to namespace the entire library, to avoid this in the future ?

[rndi]

https://github.com/obsproject/obs-studio/blob/a705030b2398598488c653238ba6f13be37a3be8/libobs/util/base.h#L86

This is merged now: #576

[boxerab]

Glad we finally understand the issue here.

[ocrete]

Congrats @boxerab @rndi @Ionic @ethouris for finally finding the root cause!

[boxerab]

https://github.com/obsproject/obs-studio/blob/a705030b2398598488c653238ba6f13be37a3be8/libobs/util/base.h#L86

@rndi the EXPORT keyword here is only used for generating windows DLLs. So, it should not affect linux builds.

[ethouris]

Global variables are evil indeed, although C++ has means to avoid problems like this. Simply global variables shall either be a static field in a class, or put under some namespace. Even if there happens to be a namespace collision, these will be already C++ symbols and shall undergo strong linkage, so the linkers shall usually find this problem. If this is a symbol in a global namespace and only one such symbol undergoes strong linkage, the linker won't be able to detect the problem.