Calculating e (base of the natural log) to high precision in Python?

Is it possible to calculate the value of the mathematical constant, *e* with high precision (2000+ decimal places) using Python? I am particularly interested in a solution either in or that...

A tool to convert MATLAB code to Python

I have a bunch of MATLAB code from my MS thesis which I now want to convert to Python (using numpy/scipy and matplotlib) and distribute as open-source. I know the similarity between MATLAB and...

How to calculate rolling / moving average using python + NumPy / SciPy?

There seems to be no function that simply calculates the moving average on numpy/scipy, leading to convoluted solutions. My question is two-fold: What's the easiest way to (correctly) implement a...

binning a dataframe in pandas in Python

Given the following dataframe in pandas: import numpy as np df = pandas.DataFrame({"a": np.random.random(100), "b": np.random.random(100), "id": np.arange(100)}) where id is an id for each point...

How can I ignore zeros when I take the median on columns of an array?

I have a simple numpy array. array([[10, 0, 10, 0], [ 1, 1, 0, 0] [ 9, 9, 9, 0] [ 0, 10, 1, 0]]) I would like to take the median of each column,...

How to center labels in histogram plot

I have a numpy array results that looks like [ 0. 2. 0. 0. 0. 0. 3. 0. 0. 0. 0. 0. 0. 0. 0. 2. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. ...

Python JSON encoder convert NaNs to null instead

I'm writing code to receive an arbitrary object (possibly nested) capable of being converted to JSON. The default behavior for Python's builtin JSON encoder is to convert NaNs to NaN, e.g....

Full-matrix approach to backpropagation in Artificial Neural Network

I am learning Artificial Neural Network (ANN) recently and have got a code working and running in Python for the same based on mini-batch training. I followed the book of Michael Nilson's Neural...

Numpy - check if elements of a array belong to another array

I have 2 numpy arrays: xarr = np.array([1.1, 1.2, 1.3, 1.4, 1.5]) y = np.array([1.1,1.2]) I want to check whether each element of xarr belongs to y or equals 1.3. If an element belongs to y,...

Randomly insert NA's values in a pandas dataframe

How can I randomly insert np.nan's in a DataFrame ? Let's say I want 10% null values inside my DataFrame. My data looks like this : df = pd.DataFrame(np.random.randn(5, 3), ...

Is there anything faster than dict()?

I need a faster way to store and access around 3GB of k:v pairs. Where k is a string or an integer and v is an np.array() that can be of different shapes. Is there any object that is faster than...

Most efficient way to forward-fill NaN values in numpy array

Example Problem As a simple example, consider the numpy array arr as defined below: import numpy as np arr = np.array([[5, np.nan, np.nan, 7, 2], [3, np.nan, 1, 8, np.nan], ...

ValueError: could not broadcast input array from shape (224,224,3) into shape (224,224)

I have a list say, temp_list with following properties : len(temp_list) = 9260 temp_list[0].shape = (224,224,3) Now, when I am converting into numpy array, x = np.array(temp_list) I am...

Can memmap pandas series. What about a dataframe?

It seems that I can memmap the underlying data for a python series by creating a mmap'd ndarray and using it to initialize the Series. def assert_readonly(iloc): try: ...

How to find first non-zero value in every column of a numpy array?

Suppose I have a numpy array of the form: arr=numpy.array([[1,1,0],[1,1,0],[0,0,1],[0,0,0]]) I want to find the indices of the first index (for every column) where the value is non-zero. So in...

Compute the Jacobian matrix in Python

import numpy as np a = np.array([[1,2,3], [4,5,6], [7,8,9]]) b = np.array([[1,2,3]]).T c = a.dot(b) #function jacobian = a # as partial derivative of c w.r.t to b...

Sampling unique column indexes for each row of a numpy array

I want to generate a fixed number of random column indexes (without replacement) for each row of a numpy array. A = np.array([[3, 5, 2, 3, 3], [1, 3, 3, 4, 5], [3, 5, 4, 2, 1], ...

Why doesn't numpy.zeros allocate all of its memory on creation? And how can I force it to?

I want to create an empty Numpy array in Python, to later fill it with values. The code below generates a 1024x1024x1024 array with 2-byte integers, which means it should take at least 2GB in...

numpy is already installed with Anaconda but I get an ImportError (DLL load failed: The specified module could not be found)

I am using the Anaconda distribution with Python 3.7. Among the packages installed, I have numpy, pandas, etc. In PyCharm IDE, I have set the Project Interpreter to be the path to the python.exe...

Conda downgrade numpy version

I need to downgrade numpy version: python -c "import numpy; print(numpy.__version__)" 1.16.4 conda install numpy==1.14.3 Collecting package metadata (current_repodata.json): done Solving...

Mapping sets of points with affine transformation

There are 2 sets of points: The points are in separate pandas dataframes (python 3), stored as the points in rows and the values of the x and y positions in columns: Centroid X µm Centroid Y...

numpy.float128 doesn't exist in windows, but is called from OpenGL

I decided to try using OpenGL VBO in Python to improve FPS. I found code, that worked perfectly fine in Linux OS (Ubuntu), but when I tried launching in Windows OS, the code resulted in a...

Return an std::vector to python as a numpy array

Using Pybind11, I am trying to pass a numpy array to c++ into a std::vector, multiply it by 2, and return this std::vector to python as a numpy array. I have achieved the first step but the third...

How to use numpy.random to generate random numbers from a certain distribution?

I am somewhat confused about how to use numpy.random to generate random values from a give distribution, say, binomial. I thought it would be import numpy as np np.random.binomial(10, 0.3,...

pd.NA vs np.nan for pandas

pd.NA vs np.nan for pandas. Which one to use with pandas and why to use? What are main advantages and disadvantages of each of them with pandas? Some sample code that uses them both: import pandas...

Random number generation following a Poisson distribution

I have prepared a code in Python to do random sampling of beam structure and looking for photons. The evolution of photons with time follows a Poisson distribution. The beam structure I am...

What does r() function mean in the return value of SymPy's dsolve?

I want to evaluate the value of phi(+oo) where phi(xi) is the solution of ODE Eq(Derivative(phi(xi), (xi, 2)), (-K + xi**2)*phi(xi)) and K is a known real variable. By dsolve, I got the...

tensorflow 2.5.0 has requirement h5py~=3.1.0, but you'll have h5py 3.3.0 which is incompatible

I am trying to install TensorFlow but getting these errors: ERROR: TensorFlow 2.5.0 has requirement h5py~=3.1.0, but you'll have h5py 3.3.0 which is incompatible. ERROR: tensorflow 2.5.0 has...

Fastest way to write numpy array in arrow format

I'm looking for fast ways to store and retrieve numpy array using pyarrow. I'm pretty satisfied with retrieval. It takes less than 1 second to extract columns from my .arrow file that contains...

Find probability of sequence conversion for sub sequences and their combinations

I have data that looks like this sno Sequence conversion 1 A-B-C-D-E-B-A 1 2 A-B-C-D ...