Grouping Parametrized Benchmarks with pytest

I'm currently benchmarking an implementation of an AVL Tree I made against a non-rebalancing binary search tree using pytest-benchmark. It seems to be working well for me so far but I've run into...

pytest and parallel program benchmarking

I'am using py.test module to benchmark two versions of my algorithm and results I'm shown are way lower that those I get when I manually run the program. First variant is reference algorithm,...

Python3 + pytest + pytest-mock: Mocks leaking into other test functions breaking assertions?

NOTE: All Details about my setup (python version, modules etc) listed at bottom of question. Apologies in advance if this issue is blatant, but i've been wrestling with it for several days now....

pytest-timeit or pytest-benchmark which one is better in terms of accuracy s?

I am testing response status code and data of flask-restful api in pytest , now I would like to test the time these end points are taking , I am considering pytest-timeit a benchmark plugin , does...

pytest fails silently when can't instantiate PyQt5.QtWidgets.QApplication

When I run the following simple test with my installation of Python 3.6.0 :: Anaconda custom (64-bit), it passes. with my installation of Python 3.6.2 :: Continuum Analytics, Inc. it fails...

using pytest.fixture on setup_method

Is it possible to use pytest.fixture on setup_method so some operation can be always finish between each testcase? I have tried to use fixture like following and the structure looks like ok. I am...

PYTEST-HTML report on test fail

My question: How to generate the html report for test script when testcase fails by pytest html plugin. As per my coverage on the topic I found this link : In this link at the bottom they have...

How do I test for speed with PyTest / tox?

For testing machine learning algorithms / repositories, I see three things that matter: Does it crash Does it have a minimum test accuracy Is it fast enough While (1) and maybe (2) is standard...

pytest-benchmark: Run setup on each benchmark iteration

I'm trying to benchmark the bundling process of our js bundles using pytest-benchmark. For accurate processing the target directory needs to empty. I've tried cleaning this on each run using the...

How to time tests with pytest?

How can I display the amount of time taken for each test executed by pytest? I've investigated pytest-timeout, pytest-timeit, and pytest-benchmark. pytest-benchmark is closest, but requires...

Adding UUID type to Cerberus leads to BAD_TYPE error

I'm attempting to add custom data types to Cerberus. The UUID class works as expected (it's a standard library class) but I'm not able to validate with the UUID type using Cerberus. Secondarily I...

pytest & pytest-benchmark: serial or parallel?

In a single file: test.py, I have 3 test functions: test1(), test2(), test3(). Does pytest and pytest-benchmark run these 3 test cases in parallel or in serial? I have 3 files: test1.py,...

Setting pytest-benchmark configurations in pytest.ini

I am trying to setup the pytest.ini configuration file such that any benchmark test runs with the "--benchmark-autosave" option as on and other tests do not. I have tried adding the option in the...

write pytest test function return value to file with pytest.hookimpl

I am looking for a way to access the return value of a test function in order to include that value in a test report file (similar to...

Exception in session-scoped pytest fixture is reported for every test that uses that fixture -- I only want to see it once

I have a heavily-used fixture that sometimes errors out and makes it impossible to run the tests that use it (this is due to hardware errors, so it's not fixable). When it fails, pytest prints the...

ModuleNotFoundError: No module named 'src'. Python 3 error

I'm facing the dreaded ModuleNotFoundError: No module named 'src' error. I've done quite some extensive reading into this problem and see that this is a common problem. I can't seem to wrap my...

Benchmark a function over different inputs (of different sizes) in python efficiently

I'm trying to benchmark a function over different inputs which have different sizes. For example I have a function something(array) that takes a numpy array with variable length as an input and a...

Cython: How to speed up recursive functions?

I am implementing a segment tree in cython and comparing it with the python implementation. The cython version seems only 1.5x times faster and I am wanting to make it even faster. Both...

Runtime statistics for parallel test function calls (with pytest-benchmark or some other plugin)

I want to use a plugin like pytest-benchmark to show me runtime statistics for the parallel calls test_valid_submission(0), ..., test_valid_submission(EXPERIMENT_SIZE). For this, I have the...