1. Forum
  2. Usenet
  3. COMP.ARCH.EMBEDDED

  • Unit testing frameworks for embedded c

    From pozz@21:1/5 to All on Wed Jan 26 13:23:15 2022
    Until now, I was not be able to arrange a full unit testing for any of
    my projects, but I want to invest some time to learn this method of development.

    There are plenty C unit testing frameworks out there. I think the most
    used are Unity[1] and Cpputest[2]. It seems to me others aren't oriented
    to embedded and/or C language (but C++).

    I read many comparisons among unit testing frameworks, but sincerely I
    can't decide which one to use.

    Do you use any? Which one and why?




    [1] http://www.throwtheswitch.org/unity

    [2] https://cpputest.github.io/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefan Reuther@21:1/5 to All on Wed Jan 26 18:10:17 2022
    Am 26.01.2022 um 13:23 schrieb pozz:
    I read many comparisons among unit testing frameworks, but sincerely I
    can't decide which one to use.

    Do you use any? Which one and why?

    None. Just <assert.h>.

    For me, the main objective for a unit test framework is to have some
    syntactic sugar for assertions and, less important, for grouping tests
    into test suites and stuff.

    Do we need assertions? It is surely helpful, if an equality assertion
    fails, to have expected and actual value presented on a golden plate. On
    the other hand, getting those values with a debugger isn't much more
    complex, and you typically fire up a debugger anyway to investigate. In
    a language such as Java or C++, you can have some higher-level
    assertions such as "assert-throws" or "assert-container-contains", which
    save quite a lot of typing. In C, the highest level you could get is
    something like "assert-string-equals", but the gain compared to plain 'assert(strcmp(...) == 0)' is vanishingly small.

    Do we need to group tests? People delight in numbers, "I have 937 unit
    tests, and of those, 936 pass, look how awesome I am!". But, seriously,
    even if one test fails, the product is broken and someone needs to
    investigate, so, for me, unit tests produce a "ok"/"nok" result, so I
    just need a binary that runs or crashes.

    Now, what I don't expect in a unit test framework proper is the whole
    area of mocking stuff. In languages like Java, how to mock something is
    pretty straightforward: you use abstract interfaces, factories, etc.,
    and to mock a file system access, you pass a mock FileSystem object. In
    C, how you can mock - if you mock at all - depends on your codebase. Do
    you have the equivalent of virtual interfaces? Do you want to play
    linker tricks? Do you want to play preprocessor tricks? That decides
    what mocking framework, if any, you can use.

    Personally, when I unit-test C, I'm testing computations and logic,
    which usually don't need mocking. Things like: put in this input, expect
    this output, from my computational kernel.


    Stefan

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From pozz@21:1/5 to All on Thu Jan 27 11:14:31 2022
    Il 26/01/2022 18:10, Stefan Reuther ha scritto:
    Am 26.01.2022 um 13:23 schrieb pozz:
    I read many comparisons among unit testing frameworks, but sincerely I
    can't decide which one to use.

    Do you use any? Which one and why?

    None. Just <assert.h>.

    For me, the main objective for a unit test framework is to have some syntactic sugar for assertions and, less important, for grouping tests
    into test suites and stuff.

    Do we need assertions? It is surely helpful, if an equality assertion
    fails, to have expected and actual value presented on a golden plate. On
    the other hand, getting those values with a debugger isn't much more
    complex, and you typically fire up a debugger anyway to investigate. In
    a language such as Java or C++, you can have some higher-level
    assertions such as "assert-throws" or "assert-container-contains", which
    save quite a lot of typing. In C, the highest level you could get is something like "assert-string-equals", but the gain compared to plain 'assert(strcmp(...) == 0)' is vanishingly small.

    Do we need to group tests? People delight in numbers, "I have 937 unit
    tests, and of those, 936 pass, look how awesome I am!". But, seriously,
    even if one test fails, the product is broken and someone needs to investigate, so, for me, unit tests produce a "ok"/"nok" result, so I
    just need a binary that runs or crashes.

    I mostly agree with you. Indeed I think it's not important which
    framework you use, but if you use and how you arrange unit testing at first.

    The big issue with C embedded projects is how you organize your build
    system to (maybe it can be an entire discussion of a new post):

    - build Release/Debug for multiple embedded targets and models of final
    product
    - build the same as above, but for host machine (simulator)
    - build and run tests on the host
    - (eventually) build and run tests on the targets

    This is more difficult for embedded because we use different
    cross-compilers for multiple targets, multiple IDEs from silicon vendors
    and so on.

    This is my main reason why migrate to a Makefile and abandon automatic
    build system of IDEs from silicon vendors.
    Starting from a custom and ubiquitous (even if cryptic and ancient)
    Makefile, I can customize the entire build system for production on real targets, simulators, tests and so on.

    However this is very difficult for me and starting from a working and
    non trivial example is a big bonus.

    I found this article[1] very very interesting in this topic. It uses
    Cpputest in examples (even if the author clearly says that the framework
    is not important).

    The author gives a pro to frameworks for code coverage report. I know
    almost anything on this, so I don't know if code coverage can be
    reported without a unit test framework.


    Now, what I don't expect in a unit test framework proper is the whole
    area of mocking stuff. In languages like Java, how to mock something is pretty straightforward: you use abstract interfaces, factories, etc.,
    and to mock a file system access, you pass a mock FileSystem object. In
    C, how you can mock - if you mock at all - depends on your codebase. Do
    you have the equivalent of virtual interfaces? Do you want to play
    linker tricks? Do you want to play preprocessor tricks? That decides
    what mocking framework, if any, you can use.

    Mocking is another argument that I will certainly explore. I will start
    testing simple units that don't have many dependecies and where I can
    use stubs or fakes, not real mocks.


    Personally, when I unit-test C, I'm testing computations and logic,
    which usually don't need mocking. Things like: put in this input, expect
    this output, from my computational kernel.

    This will be my starting point too, but I think it is a small part of a
    non trivial project.
    Critical and obscure bugs are hidden in complex units that have many dependencies that can't be replaced by simple fakes and stubs during
    tests, you need full mocks.






    [1] https://interrupt.memfault.com/blog/unit-testing-basics

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)