Importing data and variable names from a text file in Python

dynamic, genfromtxt, python, variable-assignment, variables

Solution

Instead of trying to assign names, you might think about using an associative array, which is known in Python as a `dict`, to store your variables and their values. The code could then look something like this (borrowing liberally from the `csv` docs):

import csv
with open('1.txt', 'rt') as f:
  reader = csv.reader(f, delimiter=' ', skipinitialspace=True)

  lineData = list()

  cols = next(reader)
  print(cols)

  for col in cols:
    # Create a list in lineData for each column of data.
    lineData.append(list())


  for line in reader:
    for i in xrange(0, len(lineData)):
      # Copy the data from the line into the correct columns.
      lineData[i].append(line[i])

  data = dict()

  for i in xrange(0, len(cols)):
    # Create each key in the dict with the data in its column.
    data[cols[i]] = lineData[i]

print(data)

`data` then contains each of your variables, which can be accessed via `data['varname']`.

So, for example, you could do `data['a']` to get the list `['1', '2', '3', '4']` given the input provided in your question.

I think trying to create names based on data in your document might be a rather awkward way to do this, compared to the dict-based method shown above. If you really want to do that, though, you might look into reflection in Python (a subject I don't really know anything about).

Problem

I have a text file containing simulation data (60 columns, 100k rows): ``` a b c 1 11 111 2 22 222 3 33 333 4 44 444 ``` ... where in the first row are variable names, and beneath (in columns) is the corresponding data (float type). I need to use all these variables with their data in Python for further calculations. For example, when I insert: ``` print(b) ``` I need to receive the values from the second column. I know how to import data: ``` data=np.genfromtxt("1.txt", unpack=True, skiprows = 1) ``` Assign variables "manually": ``` a,b,c=np.genfromtxt("1.txt", unpack=True, skiprows = 1) ``` But I'm having trouble with getting variable names: ``` reader = csv.reader(open("1.txt", "rt")) for row in reader: list.append(row) variables=(list[0]) ``` How can I change this code to get all variable names from the first row and assign them to the imported arrays ?

Original source