Addition of array in python with list in a list

How to sum up the below to list? a=[[1,2,3],[4,5,6],[7,8,9]] b=[[1,2,3],[4,5,6],[7,8,9]] I apply this code: Total=[x + y for x, y in zip(a, b)] So the output will be: Total=[[1,1,2,2,3,3],[4,4,5,5,6,6],[7,7,8,8,9,9]] but I wish to get Tot...
more »

2017-08-09 03:08 (3) Answers

What additional work is done by np.power?

I realised that np.power(a, b) is slower than np.exp(b * np.log(a)): import numpy as np a, b = np.random.random((2, 100000)) %timeit np.power(a, b) # best of 3: 4.16 ms per loop %timeit np.exp(b * np.log(a)) # best of 3: 1.74 ms per loop The resul...
more »

2017-08-07 11:08 (1) Answers

Pythonic way to extract 2D array from list

Say I have a list which contains 16 elements: lst=['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P'] This list represents a 4 x 4 array where all elements have been put into a 1D list. In its array form it has this form: 'A', 'B', ...
more »

2017-08-05 14:08 (3) Answers

Scipy Sparse Cumsum

Suppose I have a scipy.sparse.csr_matrix representing the values below [[0 0 1 2 0 3 0 4] [1 0 0 2 0 3 4 0]] I want to calculate the cumulative sum of non-zero values in-place, which would change the array to: [[0 0 1 3 0 6 0 10] [1 0 0 3 0 6 1...
more »

2017-08-03 21:08 (0) Answers

What does `x[False]` do in numpy?

Say I have an array x = np.arange(6).reshape(3, 2). What is the meaning of x[False], or x[np.asanyarray(False)]? Both result in array([], shape=(0, 3, 2), dtype=int64), which is unexpected. I expected to get an IndexError because of an improperly s...
more »

2017-08-03 21:08 (1) Answers

numpy dot product for tensors (3d times 2d)

Currently I use Na = (3, 2, 4) Nb = Na[1:] A = np.arange(np.prod(Na)).reshape(Na) b = np.arange(np.prod(Nb)).reshape(Nb) I want to calculate: r = np.empty((A.shape[0], A.shape[2]) for i in range(A.shape[2]): r[:, i] = np.dot(A[:, :, i], b[...
more »

2017-08-01 16:08 (1) Answers

Add n zeros to the end of an array

I want to add n zeros to an array. When your array is x, and you want to add 3 zeros at the and of an array without creating 2 arrays: x = np.array([1.0, 2.0, 1.0, 2.0, 7.0, 9.0, 1.0, 1.0, 3.0, 4.0, 10.0]) I thought this command would be helpful...
more »

2017-07-31 19:07 (1) Answers

Should I use np.absolute or np.abs?

Numpy provides both np.absolute and the alias np.abs defined via from .numeric import absolute as abs which seems to be in obvious violation of the zen of python: There should be one-- and preferably only one --obvious way to do it. So I'm g...
more »

2017-07-31 12:07 (1) Answers

Speeding up calculation of nearby groups?

I have a data frame that contains a group ID, two distance measures (longitude/latitude type measure), and a value. For a given set of distances, I want to find the number of other groups nearby, and the average values of those other groups nearby. ...
more »

2017-07-28 14:07 (1) Answers

Large Numpy Scipy CSR Matrix, row wise operation

I want to iterate over the rows of a CSR Matrix and divide each element by the sum of the row, similar to this here: numpy divide row by row sum My problem is that I'm dealing with a large matrix: (96582, 350138) And when applying the operation fr...
more »

2017-07-28 14:07 (2) Answers

Interweave two dataframes

Suppose I have two dataframes d1 and d2 d1 = pd.DataFrame(np.ones((3, 3), dtype=int), list('abc'), [0, 1, 2]) d2 = pd.DataFrame(np.zeros((3, 2), dtype=int), list('abc'), [3, 4]) d1 0 1 2 a 1 1 1 b 1 1 1 c 1 1 1 d2 3 4 a ...
more »

2017-07-26 18:07 (7) Answers

Creating Pandas dataframe from numpy array row

I have a numpy array looking like this: a = np.array([35,2,160,56,120,80,1,1,0,0,1]) Then I'm trying to transform that array into pandas dataframe with logic "one column-one value" like this: columns=['age','gender','height', 'weight','ap_hi...
more »

2017-07-26 14:07 (2) Answers

Numpy match indexing dimensions

Problem I have two numpy arrays, A and indices. A has dimensions m x n x 10000. indices has dimensions m x n x 5 (output from argpartition(A, 5)[:,:,:5]). I would like to get a m x n x 5 array containing the elements of A corresponding to indice...
more »

2017-07-25 20:07 (2) Answers

Fill in numbers in a numpy array in sets of 1024

I have a large numpy array with increasing elements as follows A = [512,2560,3584,5632,....] Elements are always spaced at least 1024 apart Let's say I need to transform that above array to the one below, where for each element of the original m...
more »

2017-07-25 00:07 (2) Answers

How to find memory leak with pandas

I have a program which repeatedly loops over a pandas data frame like below: monts = [some months] for month in months: df = original_df[original_df.month == month].copy() result = some_function(df) print(result) However, the memory which i...
more »

2017-07-24 20:07 (1) Answers

Initialize transposed numpy array

I want to use the Singular-Value-Decomposition of matrix A. If possible I would write: V, S, W.T = np.linalg.svd(A) But I can't initialise an array with its transposed. Now I have two questions: As far as I understand the python internals ther...
more »

2017-07-18 14:07 (1) Answers