Conan: Proof of Concept of Testing with multiple tooling
We want the unittest to pass, even if users don't have all tools installed. We use a bunch of them:
- Cmake, SVN, pkg-config, Visual...
We want control over which versions run by default for tools, and control of multiple versions for the same tool
memsharded
All 7 comments
First approach to this issue. Alternatives:
noseis dead, it shouldn't be considered an alternativepytest: (can rununittest)unittest
unittest
We can write our own tools on top of unittests. We would need something to add _tags_ or _marks_ to the tests and probably write our custom decorator. Not hard at first sight, but it might get complicated, custom and hard to maintain/understand.
@conantesting_attribs(['cmake', 'gcc',...])
class MyTestCase(...): ...
def conantesting_attribs(obj, attr):
# Check list of attributes against some condition (JSON, PATH,...)
condition_match = ...
if condition_match:
return lambda func: func
return unittest.skip("{!r} doesn't satisfy attributes {!r}".format(obj, attr))
pytest
pytest already provides all these functionality, with hooks and plugins. Much more powerful and better prepared to handle any situation we might imagine.
But, there are several drawbacks in order to adopt pytest. It uses a more modern approach and tests inheriting from unittest.TestCase have only very few features available (https://docs.pytest.org/en/stable/unittest.html), among those we cannot use they are listed custom hooks (probably not needed) and markers (POC needed here).
Nevertheless, a POC implementation of marks works with classes inheriting from unittest.TestCase. This works:
conftest.py
import pytest
# More about hooks: https://doc.pytest.org/en/2.7.3/plugins.html?highlight=re#pytest-hook-reference
tools_available = {
'cmake': '3.10',
'gcc': '7'
}
tools_keys = tools_available.keys()
def pytest_runtest_setup(item):
# Every mark is a required tool, some specify a version
for mark in item.iter_markers():
if mark.name not in tools_keys:
pytest.skip("required {} not satisfied".format(mark.name))
else:
version = mark.kwargs.get('version', None)
if version: # Implement version_range, regex,...
pytest.skip("required version {} {} not satisfied".format(mark.name, version))
test_module.py
import unittest
import pytest
@pytest.mark.cmake
class TestModule(unittest.TestCase):
@pytest.mark.gcc
def test_upper(self):
assert 'foo'.upper() == 'FOO'
@pytest.mark.gcc(version="4.9")
def test_upper_49(self):
assert 'foo'.upper() == 'FOO'
@pytest.mark.protoc
def test_isupper(self):
assert 'FOO'.isupper()
assert not 'Foo'.isupper()
Output running these tests:
⇒ pytest test_module.py -vv -rsx
============================================ test session starts =============================================
platform darwin -- Python 3.8.2, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /Users/jgsogo/dev/_virtualenvs/py38/bin/python
cachedir: .pytest_cache
rootdir: /Users/jgsogo/dev/conan/register-tests, configfile: pytest.ini
collected 3 items
test_module.py::TestModule::test_isupper SKIPPED [ 33%]
test_module.py::TestModule::test_upper PASSED [ 66%]
test_module.py::TestModule::test_upper_49 SKIPPED [100%]
========================================== short test summary info ===========================================
SKIPPED [1] conftest.py:17: required protoc not satisfied
SKIPPED [1] conftest.py:21: required version gcc 4.9 not satisfied
======================================== 1 passed, 2 skipped in 0.03s ========================================
Best news is that we can start using pytest without changing too much our test suite (no need to remove all unittest.TestCase inheritance), so ideally this is something that can be done in parallel: run pytest in parallel, add new marks, get rid of nose.plugin.attrs, stop using nose, consider migrating to pytest native to take advantage of more features (fixtures, server,...).
jgsogo
on 6 Oct 2020
❤2
I am loving it ❤️
Looks perfect for the need, plus seems that moving to pytest is the right thing to do to have a modern and enjoyable test suite.
memsharded
on 6 Oct 2020
Running pytest with our test suite in MacOS works (I need to upgrade the Mac, that's the reason I get some errors):
⇒ pytest conans/test
============================================ test session starts =============================================
platform darwin -- Python 3.8.2, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/jgsogo/dev/conan/conan
collected 1560 items
conans/test/conan_v2/test_cppinfo.py . [ 0%]
conans/test/conan_v2/test_default_config.py sx.. [ 0%]
conans/test/conan_v2/test_options_format.py ... [ 0%]
conans/test/conan_v2/test_profile_loader.py .. [ 0%]
conans/test/conan_v2/test_python_requires.py . [ 0%]
conans/test/conan_v2/test_pythonpath.py . [ 0%]
conans/test/conan_v2/conanfile/test_collect_libs.py . [ 0%]
conans/test/conan_v2/conanfile/test_config_method.py . [ 0%]
conans/test/conan_v2/conanfile/test_environment.py .. [ 1%]
conans/test/conan_v2/conanfile/test_package_id_method.py .. [ 1%]
conans/test/conan_v2/conanfile/test_package_method.py . [ 1%]
conans/test/conan_v2/conanfile/test_source_method.py .. [ 1%]
conans/test/conan_v2/conanfile/test_version_type.py ... [ 1%]
conans/test/conan_v2/generators/test_cpp_info.py . [ 1%]
conans/test/conan_v2/generators/test_pkg_config.py ... [ 1%]
conans/test/conan_v2/settings_model/test_cppstd.py ...... [ 2%]
conans/test/conan_v2/tools/test_tools_win.py .. [ 2%]
conans/test/functional/revisions_test.py ssssssssssssssssssssssssssssssssssssssssssss [ 5%]
conans/test/functional/sysroot_test.py . [ 5%]
conans/test/functional/test_migrations.py .... [ 5%]
conans/test/functional/tgz_macos_dot_files_test.py . [ 5%]
conans/test/functional/build_helpers/autotools_environment_test.py . [ 5%]
conans/test/functional/build_helpers/cmake_apple_test.py .................... [ 6%]
conans/test/functional/build_helpers/cmake_configs_test.py . [ 6%]
conans/test/functional/build_helpers/cmake_flags_test.py s [ 6%]
conans/test/functional/build_helpers/cmake_test.py .... [ 7%]
conans/test/functional/build_helpers/meson_test.py s [ 7%]
conans/test/functional/build_helpers/pkg_config_test.py .. [ 7%]
conans/test/functional/build_helpers/visual_environment_test.py . [ 7%]
conans/test/functional/build_requires/build_requires_test.py ................ [ 8%]
conans/test/functional/build_requires/profile_build_requires_test.py ..... [ 8%]
conans/test/functional/build_requires/profile_build_requires_testing_test.py . [ 8%]
conans/test/functional/build_requires/profile_requires_test.py . [ 8%]
conans/test/functional/cache/download_cache_test.py .... [ 9%]
conans/test/functional/cache/paths_test.py .... [ 9%]
conans/test/functional/cache/remove_old_export_sources_layout_test.py . [ 9%]
conans/test/functional/command/alias_test.py ..... [ 9%]
conans/test/functional/command/config_install_test.py ........................... [ 11%]
conans/test/functional/command/config_test.py ...... [ 11%]
conans/test/functional/command/copy_test.py ...... [ 12%]
conans/test/functional/command/create_test.py ... [ 12%]
conans/test/functional/command/export_pkg_test.py ....................... [ 14%]
conans/test/functional/command/get_test.py ........... [ 14%]
conans/test/functional/command/inspect_test.py .............. [ 15%]
conans/test/functional/command/remote_test.py ..... [ 15%]
conans/test/functional/command/remove_test.py ... [ 16%]
conans/test/functional/command/search_test.py ..ss. [ 16%]
conans/test/functional/command/source_test.py . [ 16%]
conans/test/functional/command/test_command_test.py .. [ 16%]
conans/test/functional/command/test_package_test.py .. [ 16%]
conans/test/functional/command/user_test.py ........... [ 17%]
conans/test/functional/command/download/download_test.py .... [ 17%]
conans/test/functional/command/export/export_dirty_test.py .ss [ 17%]
conans/test/functional/command/export/export_path_test.py .... [ 18%]
conans/test/functional/command/export/export_test.py .................. [ 19%]
conans/test/functional/command/export/exports_method_test.py ........... [ 20%]
conans/test/functional/command/info/info_command_test.py . [ 20%]
conans/test/functional/command/info/info_folders_test.py .....sss. [ 20%]
conans/test/functional/command/info/info_test.py ..... [ 21%]
conans/test/functional/command/install/install_cwd_test.py . [ 21%]
conans/test/functional/command/install/install_test.py . [ 21%]
conans/test/functional/command/upload/upload_complete_test.py . [ 21%]
conans/test/functional/command/upload/upload_test.py .... [ 21%]
conans/test/functional/conan_api/config_test.py . [ 21%]
conans/test/functional/conan_api/manifests_arguments_test.py ......... [ 22%]
conans/test/functional/conan_api/two_conan_creates_test.py .s [ 22%]
conans/test/functional/conan_build_info/test_build_info_creation.py ..... [ 22%]
conans/test/functional/conan_build_info/test_build_info_extraction.py .... [ 22%]
conans/test/functional/conanfile/conanfile_extend_test.py .. [ 22%]
conans/test/functional/conanfile/conanfile_helpers_test.py .. [ 23%]
conans/test/functional/conanfile/exception_printing_test.py ... [ 23%]
conans/test/functional/conanfile/generators_list_test.py ... [ 23%]
conans/test/functional/conanfile/in_local_cache_test.py . [ 23%]
conans/test/functional/conanfile/invalid_configuration_test.py ...... [ 23%]
conans/test/functional/conanfile/no_copy_source_test.py .. [ 24%]
conans/test/functional/conanfile/runner_test.py . [ 24%]
conans/test/functional/conanfile/same_userchannel_test.py ..... [ 24%]
conans/test/functional/conanfile/user_info_test.py . [ 24%]
conans/test/functional/configuration/compressed_symlinks_test.py s [ 24%]
conans/test/functional/configuration/conan_trace_file_test.py .. [ 24%]
conans/test/functional/configuration/conf_default_settings_test.py . [ 24%]
conans/test/functional/configuration/default_profile_test.py .. [ 24%]
conans/test/functional/configuration/invalid_settings_test.py .. [ 25%]
conans/test/functional/configuration/profile_test.py ...... [ 25%]
conans/test/functional/configuration/proxies_conf_test.py .... [ 25%]
conans/test/functional/configuration/python_version_test.py ss [ 25%]
conans/test/functional/configuration/registry_test.py . [ 25%]
conans/test/functional/configuration/requester_test.py ...... [ 26%]
conans/test/functional/configuration/required_version_test.py ..... [ 26%]
conans/test/functional/cross_building/build_helper_test.py . [ 26%]
conans/test/functional/cross_building/build_requires_from_profile_test.py ... [ 26%]
conans/test/functional/cross_building/test_package_test.py .. [ 26%]
conans/test/functional/cross_building/user_info_test.py .. [ 27%]
conans/test/functional/cross_building/graph/breq_of_breq_test.py . [ 27%]
conans/test/functional/cross_building/graph/build_requires_in_profile_test.py .. [ 27%]
conans/test/functional/cross_building/graph/build_requires_in_recipe_test.py . [ 27%]
conans/test/functional/cross_building/graph/no_way_back_to_host_test.py . [ 27%]
conans/test/functional/cross_building/graph/protobuf_test.py .. [ 27%]
conans/test/functional/cross_building/graph/protoc_basic_test.py .. [ 27%]
conans/test/functional/cross_building/graph/protoc_with_gtest_test.py .. [ 27%]
conans/test/functional/cross_building/profile_access/access_other_context_test.py . [ 27%]
conans/test/functional/deprecated/test_deprecated.py ... [ 28%]
conans/test/functional/editable/consume_header_only_test.py .... [ 28%]
conans/test/functional/editable/consume_settings_and_options_test.py ........ [ 28%]
conans/test/functional/editable/editable_add_test.py ...... [ 29%]
conans/test/functional/editable/editable_remove_test.py .. [ 29%]
conans/test/functional/editable/forbidden_commands_test.py ....... [ 29%]
conans/test/functional/editable/graph_related_test.py .......... [ 30%]
conans/test/functional/editable/layouts_test.py ........ [ 30%]
conans/test/functional/editable/transitive_editable_test.py . [ 30%]
conans/test/functional/editable/commands/info_on_child_test.py ...... [ 31%]
conans/test/functional/editable/commands/info_test.py ...... [ 31%]
conans/test/functional/editable/commands/inspect_test.py . [ 31%]
conans/test/functional/environment/apply_environment_test.py ......... [ 32%]
conans/test/functional/environment/run_environment_test.py ....s [ 32%]
conans/test/functional/generators/cmake_apple_frameworks_test.py ...F....F... [ 33%]
conans/test/functional/generators/cmake_find_package_multi_test.py . [ 33%]
conans/test/functional/generators/cmake_generator_test.py s.s [ 33%]
conans/test/functional/generators/cmake_skip_rpath_test.py . [ 33%]
conans/test/functional/generators/cmake_test.py s.. [ 33%]
conans/test/functional/generators/deploy_test.py ..... [ 34%]
conans/test/functional/generators/generators_test.py ...... [ 34%]
conans/test/functional/generators/json_test.py .. [ 34%]
conans/test/functional/generators/link_order_test.py .F.F.F.F [ 35%]
conans/test/functional/generators/markdown_test.py . [ 35%]
conans/test/functional/generators/multi_generators_test.py ... [ 35%]
conans/test/functional/generators/premake_test.py ss [ 35%]
conans/test/functional/generators/qbs_test.py . [ 35%]
conans/test/functional/generators/virtualbuildenv_test.py s [ 35%]
conans/test/functional/generators/virtualenv_test.py ....ssssssss. [ 36%]
conans/test/functional/generators/virtualenv_windows_bash_test.py s [ 36%]
conans/test/functional/generators/components/propagate_specific_components_test.py ...... [ 37%]
conans/test/functional/generators/package_info/deps_cpp_info_test.py . [ 37%]
conans/test/functional/graph/alias_test.py ... [ 37%]
conans/test/functional/graph/conflict_diamond_test.py .... [ 37%]
conans/test/functional/graph/graph_manager_test.py .................................. [ 39%]
conans/test/functional/graph/package_id_modes_test.py .. [ 39%]
conans/test/functional/graph/private_deps_test.py ...... [ 40%]
conans/test/functional/graph/require_override_test.py .. [ 40%]
conans/test/functional/graph/transitive_deps_test.py .. [ 40%]
conans/test/functional/graph/version_range_no_userchannel_test.py ... [ 40%]
conans/test/functional/graph/version_range_override_test.py ... [ 40%]
conans/test/functional/graph_lock/build_order_test.py .... [ 41%]
conans/test/functional/graph_lock/graph_lock_build_requires_test.py ...... [ 41%]
conans/test/functional/graph_lock/graph_lock_ci_test.py .sss....... [ 42%]
conans/test/functional/graph_lock/graph_lock_test.py ss... [ 42%]
conans/test/functional/hooks/test_post_export.py . [ 42%]
conans/test/functional/hooks/test_post_package.py .. [ 42%]
conans/test/functional/options/options_test.py . [ 42%]
conans/test/functional/options/test_package_config_test.py . [ 42%]
conans/test/functional/package_id/package_id_requires_modes_test.py .............. [ 43%]
conans/test/functional/provides/test_build_requires_conflicts.py ......... [ 44%]
conans/test/functional/provides/test_conditional_provides.py . [ 44%]
conans/test/functional/provides/test_requires_conflicts.py .... [ 44%]
conans/test/functional/provides/test_requires_private.py .. [ 44%]
conans/test/functional/py_requires/python_requires_test.py .... [ 45%]
conans/test/functional/python_requires/python_requires_test.py .... [ 45%]
conans/test/functional/python_requires/test_export_sources.py .. [ 45%]
conans/test/functional/remote/download_retries_test.py .. [ 45%]
conans/test/functional/remote/download_test.py . [ 45%]
conans/test/functional/remote/multi_remote_checks_test.py ..... [ 46%]
conans/test/functional/remote/retry_test.py ..... [ 46%]
conans/test/functional/remote/server_error_test.py ... [ 46%]
conans/test/functional/remote/token_refresh_test.py .. [ 46%]
conans/test/functional/scm/scm_test.py ........................................... [ 49%]
conans/test/functional/scm/test_command_export.py ...... [ 49%]
conans/test/functional/scm/test_fields_validation.py .... [ 50%]
conans/test/functional/scm/test_git_shallow.py ............ [ 50%]
conans/test/functional/scm/test_local_modified.py .. [ 50%]
conans/test/functional/scm/test_scm_to_conandata.py ..... [ 51%]
conans/test/functional/scm/test_url_auto.py . [ 51%]
conans/test/functional/scm/test_verify_ssl.py .... [ 51%]
conans/test/functional/scm/issues/test_svn_tag.py . [ 51%]
conans/test/functional/scm/workflows/test_conanfile_in_repo_root.py ........ [ 52%]
conans/test/functional/scm/workflows/test_conanfile_in_subfolder.py ........ [ 52%]
conans/test/functional/scm/workflows/test_scm_subfolder.py ........ [ 53%]
conans/test/functional/settings/built_type_setting_test.py . [ 53%]
conans/test/functional/settings/conan_settings_preprocessor_test.py .. [ 53%]
conans/test/functional/settings/libcxx_setting_test.py .. [ 53%]
conans/test/functional/settings/per_package_settings_test.py .. [ 53%]
conans/test/functional/settings/settings_override_test.py .... [ 53%]
conans/test/functional/settings/cppstd/compiler_cppstd_test.py .............. [ 54%]
conans/test/functional/settings/cppstd/default_cppstd_test.py ........ [ 55%]
conans/test/functional/symlinks/symlink_package_test.py . [ 55%]
conans/test/functional/symlinks/symlinks_test.py .. [ 55%]
conans/test/functional/templates/test_user_overrides.py ... [ 55%]
conans/test/functional/toolchain/test_basic.py ... [ 55%]
conans/test/functional/toolchain/test_cmake.py ssss... [ 56%]
conans/test/functional/toolchain/test_make.py sssss [ 56%]
conans/test/functional/toolchain/test_msbuild.py sss [ 56%]
conans/test/functional/tools/cppstd_minimum_version_test.py .... [ 57%]
conans/test/functional/tools/fix_symlinks_test.py .. [ 57%]
conans/test/functional/ui/json_output_test.py .... [ 57%]
conans/test/functional/workspace/graph_test.py . [ 57%]
conans/test/functional/workspace/workspace_test.py ... [ 57%]
conans/test/integration/manifest_validation_test.py .... [ 58%]
conans/test/unittests/client/cache_test.py .. [ 58%]
conans/test/unittests/client/conan_output_test.py ...................... [ 59%]
conans/test/unittests/client/conanfile_loader_test.py ......... [ 60%]
conans/test/unittests/client/optimize_conanfile_load_test.py . [ 60%]
conans/test/unittests/client/packager_test.py . [ 60%]
conans/test/unittests/client/remote_manager_test.py . [ 60%]
conans/test/unittests/client/userio_test.py . [ 60%]
conans/test/unittests/client/cmd/ast_replacement_scm_test.py ............ [ 61%]
conans/test/unittests/client/cmd/export_test.py .... [ 61%]
conans/test/unittests/client/cmd/package_copier_test.py . [ 61%]
conans/test/unittests/client/cmd/export/test_capture_export_scm_data.py ..... [ 61%]
conans/test/unittests/client/cmd/export/test_update_revision_in_metadata.py .... [ 62%]
conans/test/unittests/client/command/parse_arguments_test.py .................... [ 63%]
conans/test/unittests/client/conf/config_installer/test_install_folder.py . [ 63%]
conans/test/unittests/client/conf/config_installer/test_process_folder.py . [ 63%]
conans/test/unittests/client/conf/detect/detected_os_test.py .......... [ 64%]
conans/test/unittests/client/conf/detect/test_gcc_compiler.py ... [ 64%]
conans/test/unittests/client/file_copier/file_copier_test.py . [ 64%]
conans/test/unittests/client/file_copier/test_report_copied_files.py . [ 64%]
conans/test/unittests/client/generators/generator_filter_error_test.py . [ 64%]
conans/test/unittests/client/generators/premake_test.py . [ 64%]
conans/test/unittests/client/generators/virtualbuildenv_test.py ... [ 64%]
conans/test/unittests/client/generators/virtualenv_test.py .... [ 65%]
conans/test/unittests/client/generators/virtualrunenv_test.py ... [ 65%]
conans/test/unittests/client/generators/txt/test_content.py .... [ 65%]
conans/test/unittests/client/generators/txt/test_dump_load.py .. [ 65%]
conans/test/unittests/client/generators/txt/test_load.py . [ 65%]
conans/test/unittests/client/graph/build_mode_test.py ............. [ 66%]
conans/test/unittests/client/graph/deps_graph_test.py . [ 66%]
conans/test/unittests/client/graph/test_grapher.py ... [ 66%]
conans/test/unittests/client/graph/test_range_resolver.py ....... [ 67%]
conans/test/unittests/client/graph/version_ranges_graph_test.py .... [ 67%]
conans/test/unittests/client/profile_loader/compiler_cppstd_test.py ......... [ 68%]
conans/test/unittests/client/profile_loader/profile_loader_test.py .... [ 68%]
conans/test/unittests/client/recorder/action_recorder_test.py . [ 68%]
conans/test/unittests/client/rest/downloader_test.py ........ [ 68%]
conans/test/unittests/client/rest/response_test.py .. [ 68%]
conans/test/unittests/client/rest/uploader_test.py ..... [ 69%]
conans/test/unittests/client/rest/rest_client_v1/test_get_package_manifest.py . [ 69%]
conans/test/unittests/client/rest/rest_client_v2/test_get_package_manifest.py . [ 69%]
conans/test/unittests/client/source/test_run_scm.py ... [ 69%]
conans/test/unittests/client/toolchain/test_make.py ssss [ 69%]
conans/test/unittests/client/tools/cppstd_required_test.py ................................. [ 71%]
conans/test/unittests/client/tools/net_test.py ... [ 72%]
conans/test/unittests/client/tools/system_pm_test.py .. [ 72%]
conans/test/unittests/client/tools/test_env.py ....... [ 72%]
conans/test/unittests/client/tools/test_files.py ... [ 72%]
conans/test/unittests/client/tools/test_version.py ......... [ 73%]
conans/test/unittests/client/tools/intel/compilervars_test.py ....F [ 73%]
conans/test/unittests/client/tools/os_info/osinfo_test.py ............ [ 74%]
conans/test/unittests/client/tools/oss/cross_building_test.py ...... [ 75%]
conans/test/unittests/client/tools/oss/get_cross_building_settings_test.py .......... [ 75%]
conans/test/unittests/client/tools/scm/test_git.py ....................... [ 77%]
conans/test/unittests/client/tools/scm/test_scm_base.py ...... [ 77%]
conans/test/unittests/client/tools/scm/test_svn.py ................................................... [ 80%]
[ 80%]
conans/test/unittests/client/tools/settings/stdcpp_library_test.py . [ 80%]
conans/test/unittests/client/tools/win/vcvars_arch_test.py sssss [ 81%]
conans/test/unittests/client/tools/win/vcvars_clangcl_test.py sss [ 81%]
conans/test/unittests/client/tools/win/vcvars_store_test.py ssss [ 81%]
conans/test/unittests/client/util/files_test.py .. [ 81%]
conans/test/unittests/client/util/files/decode_text_test.py ................. [ 82%]
conans/test/unittests/client/util/files/load_test.py .... [ 83%]
conans/test/unittests/model/conanfile_test.py .. [ 83%]
conans/test/unittests/model/env_test.py ... [ 83%]
conans/test/unittests/model/info_test.py .. [ 83%]
conans/test/unittests/model/manifest_test.py . [ 83%]
conans/test/unittests/model/options_test.py .............. [ 84%]
conans/test/unittests/model/other_settings_test.py . [ 84%]
conans/test/unittests/model/package_metadata_test.py .. [ 84%]
conans/test/unittests/model/ref_test.py ..... [ 85%]
conans/test/unittests/model/settings_test.py ........... [ 85%]
conans/test/unittests/model/transitive_reqs_test.py ....................................... [ 88%]
conans/test/unittests/model/version_test.py . [ 88%]
conans/test/unittests/model/build_info/components_test.py ...... [ 88%]
conans/test/unittests/model/build_info/cppflags_test.py .. [ 88%]
conans/test/unittests/model/build_info/deps_cpp_info_test.py . [ 88%]
conans/test/unittests/model/conan_file/create_requirements_test.py ..... [ 89%]
conans/test/unittests/model/editable_layout/apply_test.py .. [ 89%]
conans/test/unittests/model/editable_layout/load_data_test.py . [ 89%]
conans/test/unittests/model/editable_layout/parse_test.py .. [ 89%]
conans/test/unittests/model/info/vs_toolset_compatible_test.py .............. [ 90%]
conans/test/unittests/model/scm/detect_repo_test.py ... [ 90%]
conans/test/unittests/model/scm/scm_data_test.py ..... [ 90%]
conans/test/unittests/search/binary_html_table_test.py ...... [ 91%]
conans/test/unittests/search/search_query_parse_test.py .. [ 91%]
conans/test/unittests/server/conan_server_config_parser_test.py .. [ 91%]
conans/test/unittests/server/conf_test.py .. [ 91%]
conans/test/unittests/server/revision_list_test.py ... [ 91%]
conans/test/unittests/server/service/service_test.py ........ [ 92%]
conans/test/unittests/source/merge_directories_test.py .. [ 92%]
conans/test/unittests/tools/files_patch_test.py ............ [ 93%]
conans/test/unittests/util/android_test.py . [ 93%]
conans/test/unittests/util/apple_test.py ....... [ 93%]
conans/test/unittests/util/client_conf_test.py ............ [ 94%]
conans/test/unittests/util/conanfile_tools_test.py .. [ 94%]
conans/test/unittests/util/detect_test.py .... [ 94%]
conans/test/unittests/util/detected_architecture_test.py ................... [ 96%]
conans/test/unittests/util/env_reader_test.py ... [ 96%]
conans/test/unittests/util/files_extract_wildcard_test.py . [ 96%]
conans/test/unittests/util/hashed_path_test.py ... [ 96%]
conans/test/unittests/util/msvs_toolset_test.py ssssssssssssss [ 97%]
conans/test/unittests/util/pkg_config_test.py ... [ 97%]
conans/test/unittests/util/tools_test.py ...... [ 98%]
conans/test/unittests/util/unix_path_test.py s.sss..ss [ 98%]
conans/test/unittests/util/which_test.py .... [ 98%]
conans/test/unittests/util/xz_test.py ..s.. [ 99%]
conans/test/unittests/util/zip_permissions_test.py . [ 99%]
conans/test/unittests/util/files/tar_extract_test.py s [ 99%]
conans/test/unittests/util/files/test_dirty.py .... [ 99%]
conans/test/unittests/util/files/test_remove.py ... [ 99%]
conans/test/unittests/util/misc/test_make_tuple.py ... [100%]
So we can first move to pytest and then add the magic and fun we want.
jgsogo
on 6 Oct 2020
My test suite here with nose is counting 3152 tests, way more than your 1560. Are you skipping some tests already in the above? It doesn't seem so.
memsharded
on 6 Oct 2020
This is because test discovery, some of our tests don't follow standard naming test_*, we need to customize pytest invocation (or _fix_ those tests)
jgsogo
on 6 Oct 2020
Better fix those tests, right? Lets standardize
memsharded
on 6 Oct 2020
🚀1
👍1
This PoC was successfully done.
The test suite is already running pytest, nose has been removed, tests are marked with pytest.mark to annotate different tools, and CI is running with it.
There will be continued work improving the CI pipeline, testing with different versions of the tools, etc, but that will be in other issues and PRs, closing this one now.
memsharded
on 7 Jan 2021
Was this page helpful?
0 / 5 - 0 ratings
Related issues
kwint
·
3Comments
uilianries
·
3Comments
zlalanne
·
3Comments
db4
·
3Comments
Adnn
·
3Comments
Most helpful comment
Better fix those tests, right? Lets standardize