Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migration and road map from the deprecated testing API to the new ZTEST API. #47002

Closed
aaronemassey opened this issue Jun 28, 2022 · 5 comments
Closed

Migration and road map from the deprecated testing API to the new ZTEST API. #47002

aaronemassey opened this issue Jun 28, 2022 · 5 comments
Labels
area: Tests Issues related to a particular existing or missing test Meta A collection of features, enhancements or bugs

Comments

@aaronemassey
Copy link
Member

aaronemassey commented Jun 28, 2022 •
edited by yerabolu

Migration and road map from deprecated testing API to the current API.

Each test suite should be refactored to the new ZTEST API.

Some example API migrations

Migration status

The list below was initially generated by finding tests without a ZTEST_SUITE
macro usage and roughly categorizing them with respect to their directories and
tested features. For example, tests/application_development and
tests/arch/arm contain unmigrated tests.

Simple migration workflow

This does not cover all migration cases nor the entirety of the ZTEST API. This
is just a simple workflow get started on simple tests.

  • Replace ztest_test_suite() and its containing function with a
    ZTEST_SUITE() defined here with documented
    parameters    .
  • Replace prior test functions invoking zassert with ZTEST or ZTEST_USER.
  • Add CONFIG_ZTEST_NEW_API=y to the test's prj.conf.

Difficulty or questions

Usually, a single test's migration shouldn't require more than 10 minutes. If
you're having difficulty porting a test or understanding how to use the new
ZTEST API, then make a comment on this issue.

Some suggestions

  • Try the new CONFIG_ZTEST_SHUFFLE to check if the ported test functions are self-contained/independent.
  • Keep the old CONFIG_MULTITHREADING as the old test cases.
  • Do not have a “test_” prefix in the test suite name.
  • One test suite per commit. One PR to hold all related commits.
@aaronemassey aaronemassey added the Enhancement Changes/Updates/Additions to existing features label Jun 28, 2022
@cfriedt cfriedt mentioned this issue Jun 29, 2022
Closed
This was referenced Jun 29, 2022
Merged
Merged
@yerabolu yerabolu mentioned this issue Jun 30, 2022
Merged
@nashif
Copy link
Member

nashif commented Jun 30, 2022

for all of those working on this, do not have a test_ prefix in the testsuite name. This is just confusing and redundant.

@hakehuang
Copy link
Collaborator

@aaronemassey , I have a problem here, there used to be some test fucntions are registerred as shall command see below code

extern void test_dma_m2m_chan0_burst8(void);
extern void test_dma_m2m_chan1_burst8(void);
extern void test_dma_m2m_chan0_burst16(void);
extern void test_dma_m2m_chan1_burst16(void);

#ifdef CONFIG_SHELL
TC_CMD_DEFINE(test_dma_m2m_chan0_burst8)
TC_CMD_DEFINE(test_dma_m2m_chan1_burst8)
TC_CMD_DEFINE(test_dma_m2m_chan0_burst16)
TC_CMD_DEFINE(test_dma_m2m_chan1_burst16)

SHELL_CMD_REGISTER(test_dma_m2m_chan0_burst8, NULL, NULL,
			TC_CMD_ITEM(test_dma_m2m_chan0_burst8));
SHELL_CMD_REGISTER(test_dma_m2m_chan1_burst8, NULL, NULL,
			TC_CMD_ITEM(test_dma_m2m_chan1_burst8));
SHELL_CMD_REGISTER(test_dma_m2m_chan0_burst16, NULL, NULL,
			TC_CMD_ITEM(test_dma_m2m_chan0_burst16));
SHELL_CMD_REGISTER(test_dma_m2m_chan1_burst16, NULL, NULL,
			TC_CMD_ITEM(test_dma_m2m_chan1_burst16));
#endif

shall we create a marco for ZTEST_SH which looks like below

#define Z_TEST_WITH_SH(suite, fn, t_options, use_fixture)                                                  \
	static void _##suite##_##fn##_wrapper(void *data);                                         \
	static void suite##_##fn(                                                                  \
		COND_CODE_1(use_fixture, (struct suite##_fixture *fixture), (void)));              \
	static STRUCT_SECTION_ITERABLE(ztest_unit_test, z_ztest_unit_test_##suite##_##fn) = {      \
		.test_suite_name = STRINGIFY(suite),                                               \
		.name = STRINGIFY(fn),                                                             \
		.test = (_##suite##_##fn##_wrapper),                                               \
		.thread_options = t_options,                                                       \
	};                                                                                         \
	static void _##suite##_##fn##_wrapper(void *data)                                          \
	{                                                                                          \
		COND_CODE_1(use_fixture, (suite##_##fn((struct suite##_fixture *)data);),          \
			    (ARG_UNUSED(data); suite##_##fn();))                                   \
	}                                                                                          \
       TC_CMD_DEFINE(suite##_##fn)                                                           \
       SHELL_CMD_REGISTER(suite##_##fn, NULL, NULL, TC_CMD_ITEM(suite##_##fn));  \
	static inline void suite##_##fn(                                                           \
		COND_CODE_1(use_fixture, (struct suite##_fixture *fixture), (void)))

@carlocaione carlocaione mentioned this issue Jul 6, 2022
Merged
yerabolu added a commit to yerabolu/zephyr that referenced this issue Jul 6, 2022
@yerabolu
tests: lib: cmsis_nn: move to new ztest API
6c9ca1a 
Move lib tests to use new ztest API

Part of zephyrproject-rtos#47002 effort

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
@aaronemassey
Copy link
Member Author

aaronemassey commented Jul 6, 2022 •
edited

@hakehuang

@aaronemassey , I have a problem here, there used to be some test fucntions are registerred as shall command see below code

Looking at this still, but I'm a bit confused as to actually how the SHELL_COMMAND_REGISTER macro is actually being leveraged in this test. It looks like the test is simply running the test functions themselves without respect to the shell command macro. Maybe we can discuss this at the next working group meeting (tomorrow).

Apologies for late response. Been moving.

nashif pushed a commit that referenced this issue Jul 7, 2022
@yerabolu @nashif
tests: lib: cmsis_nn: move to new ztest API
fe95263 
Move lib tests to use new ztest API

Part of #47002 effort

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
@yerabolu yerabolu mentioned this issue Jul 7, 2022
Merged
yerabolu added a commit to yerabolu/zephyr that referenced this issue Jul 7, 2022
@yerabolu
tests: lib: Move to new ztest API
38b5112 
Move lib tests to use new ztest API

Part of zephyrproject-rtos#47002 effort

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
@hakehuang hakehuang mentioned this issue Jul 8, 2022
Merged
nashif pushed a commit that referenced this issue Jul 9, 2022
@yerabolu @nashif
tests: lib: Move to new ztest API
c34edc3 
Move lib tests to use new ztest API

Part of #47002 effort

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
@simhein simhein mentioned this issue Jul 11, 2022
Merged
@simhein
Copy link
Contributor

simhein commented Jul 11, 2022 •
edited

@aaronemassey subsys/fs is wrong checked in the issue. See: https://github.com/zephyrproject-rtos/zephyr/tree/main/tests/subsys/fs. I started working on it and added my name to the task.

hakehuang added a commit to nxp-zephyr/zephyr that referenced this issue Jul 12, 2022
@hakehuang
tests: dma: upgrade dma test to new ztest api
808ad9f 
1. config ztest_new_api and update the testcase defines

2. remove shell interactive test cases, twister has another plan

3. add sg case support for kinetis and rt10xx platfrom

related: zephyrproject-rtos#47002

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
hakehuang added a commit to nxp-zephyr/zephyr that referenced this issue Jul 12, 2022
@hakehuang
tests: adc: migrate adc to ztest new
b7ff81c 
move to ztest_new api for adc tests.
some naming changes as well

related: zephyrproject-rtos#47002

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
This was referenced Jul 12, 2022
Merged
Merged
mmahadevan108 pushed a commit that referenced this issue Jul 29, 2022
@hakehuang @mmahadevan108
tests: dma: upgrade dma test to new ztest api
5409480 
1. config ztest_new_api and update the testcase defines

2. remove shell interactive test cases, twister has another plan

3. add sg case support for kinetis and rt10xx platfrom

related: #47002

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
mmahadevan108 pushed a commit that referenced this issue Jul 29, 2022
@hakehuang @mmahadevan108
tests: adc: migrate adc to ztest new
ee40f97 
move to ztest_new api for adc tests.
some naming changes as well

related: #47002

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
jeremybettis pushed a commit to jeremybettis/zephyr that referenced this issue Aug 4, 2022
@hakehuang @jeremybettis
tests: dma: upgrade dma test to new ztest api
79cf774 
1. config ztest_new_api and update the testcase defines

2. remove shell interactive test cases, twister has another plan

3. add sg case support for kinetis and rt10xx platfrom

related: zephyrproject-rtos#47002

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
jeremybettis pushed a commit to jeremybettis/zephyr that referenced this issue Aug 4, 2022
@hakehuang @jeremybettis
tests: adc: migrate adc to ztest new
6eaf138 
move to ztest_new api for adc tests.
some naming changes as well

related: zephyrproject-rtos#47002

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
simhein added a commit to simhein/zephyr that referenced this issue Aug 12, 2022
@simhein
tests: subsys/fs/nvs: Move to new ztest api
cd20148 
Convert nvs tests to use new ztest API

related: zephyrproject-rtos#47002

Signed-off-by: Simon Hein <SHein@baumer.com>
fabiobaltieri pushed a commit that referenced this issue Aug 15, 2022
@simhein @fabiobaltieri
tests: subsys/fs/nvs: Move to new ztest api
f22b9b9 
Convert nvs tests to use new ztest API

related: #47002

Signed-off-by: Simon Hein <SHein@baumer.com>
@yerabolu yerabolu mentioned this issue Aug 18, 2022
Merged
@yerabolu yerabolu mentioned this issue Sep 12, 2022
Closed
yerabolu added a commit to yerabolu/zephyr that referenced this issue Sep 15, 2022
@yerabolu
tests: lib: mem_alloc: migrating mem_alloc tests to new ztest API
4ad71f7 
Migrating mem_alloc tests to new ztest API.

Part of zephyrproject-rtos#47002 effort

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
@henrikbrixandersen henrikbrixandersen added area: Tests Issues related to a particular existing or missing test Meta A collection of features, enhancements or bugs labels Nov 17, 2022
@nashif nashif removed the Enhancement Changes/Updates/Additions to existing features label Nov 19, 2022
@gopiotr gopiotr mentioned this issue Jun 12, 2023
Closed
@JordanYates
Copy link
Collaborator

I feel like this can be closed with the deprecation of the old API in v3.4

@zephyrproject-rtos zephyrproject-rtos deleted a comment from shuai1x Jul 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area: Tests Issues related to a particular existing or missing test Meta A collection of features, enhancements or bugs
Projects
Zephyr Test Framework
Status: Done
Milestone
No milestone
Development

No branches or pull requests
6 participants
@nashif @hakehuang @henrikbrixandersen @simhein @JordanYates @aaronemassey