mdvtools.tests.test_bad_data
Functions
Test what happens with NaN, Infinity... |
Module Contents
- mdvtools.tests.test_bad_data.test_bad_data()[source]
Test what happens with NaN, Infinity… Previously, although there was code for filtering na = na[~numpy.isnan(na)], this didn’t help if there also happened to be Infinity.
The json.dumps default behaviour of allow_nan=True would then cheerfully output non-compliant JSON, which contrary to what the Python documentation states is not, as of this writing (2024-02-05), compatible with any standard JavaScript based decoders I tested (let alone ‘most’).
We now use allow_nan=False, so the user would be alerted at project creation time rather than runtime, but also correct the error earlier in the process so that the particular case of isinf is handled.
If there are no valid numbers at all in a numeric column, then an exception will be thrown, in this example we catch that and as of now, end up with a datasource with 0 columns.