saving numpy matrix in text file with indices

Question

I am trying to save a 2D matrix A to text file for each non-zero entry. I would like to save it in the below format,

For the (i,j)-th entry, one line as

row[i]::column[j]::A[i,j]

where row and column are 1D numpy array corresponding to rows and column indices

I tried,

np.savetxt(rel_file,(row,column,A),fmt='%d',delimiter="::",newline="\n");

But, since shape mismatches, i get error. I do not want to iterate over each row and column indices in a loop (I think it is too time consuming, i have 5000*5000 matrix).


Show source
| arrays   | save   | python-2.7   | matrix   | numpy   2016-12-27 23:12 2 Answers

Answers to saving numpy matrix in text file with indices ( 2 )

  1. 2016-12-28 00:12

    In my first answer I display all values; with just the nonzeros it's even easier to create the column array. np.where does the trick

    In [1548]: I,J = np.nonzero(A)    # np.where 
    In [1549]: Acol = np.column_stack((I,J,A[I,J]))
    

    ==========

    For all values:

    Here's a start; it is iterative, but as I commented, so is np.savetxt.

    In [1523]: A=np.arange(25).reshape(5,5)
    
    In [1526]: list(np.ndenumerate(A))
    Out[1526]: 
    [((0, 0), 0),
     ((0, 1), 1),
     ((0, 2), 2),
     ((0, 3), 3),
     ((0, 4), 4),
     ((1, 0), 5),
    
    In [1528]: with open('txt','w') as f:
          ...:     for (i,j),v in np.ndenumerate(A):
          ...:         f.write('%d::%d::%d\n'%(i,j,v))
          ...:         
    In [45]: cat txt
    0::0::0
    0::1::1
    0::2::2
    0::3::3
    0::4::4
    1::0::5
    

    We could make a (n,3) array consisting of the indices and flattened values of A. But savetxt would still iterate on it, effectively:

    with open(...) as f:
       for row in Acol:
          f.write(fmt % tuple(row))
    

    Here's one way of making this 3 column array:

    In [1535]: I,J = np.meshgrid(np.arange(A.shape[0]),np.arange(A.shape[1]),indexin
          ...: g='ij')
    In [1536]: I
    Out[1536]: 
    array([[0, 0, 0, 0, 0],
           [1, 1, 1, 1, 1],
           [2, 2, 2, 2, 2],
           [3, 3, 3, 3, 3],
           [4, 4, 4, 4, 4]])
    In [1537]: Acol=np.column_stack((I.ravel(),J.ravel(),A.ravel()))
    In [1538]: Acol
    Out[1538]: 
    array([[ 0,  0,  0],
           [ 0,  1,  1],
           [ 0,  2,  2],
           [ 0,  3,  3],
           [ 0,  4,  4],
           ....
    
  2. 2016-12-28 00:12

    Heres a simple method that manually reshapes the original array to be a very long array consisting of row, col, value. Just filter the rows for non-zero and it should work. The below example works with this random 50,50 array.

    a = np.random.randint(0,5, (50, 50))
    
    rows = np.repeat(np.arange(50), 50)
    cols = np.tile(np.arange(50), 50)
    
    data = np.column_stack((rows, cols, a.flatten()))
    
    data_filtered = data[data[:, 2] != 0]
    np.savetxt('temp.txt',data_filtered,fmt='%d',delimiter="::",newline="\n");
    

Leave a reply to - saving numpy matrix in text file with indices

◀ Go back