Import from CSV formatting issue

Question

I have two lists of data here: dm and dmHalo (see attached image). Theses are the same lists, the only difference is that dm was exported to a csv file and then reimported, while dmHalo was generated in the notebook session.

When I go to run an interpolation on the data set the interpolation does not recognize the csv imported data, as seen in the error message. I assume this is due to a change in formatting, but I am at a loss at how to revert to the native format.

Does anyone have an idea of how to do this?

Thanks you for your help!

Have you check with Dimensions[] that both object have the same dimensions? — Drod, May 20 '21 at 20:15
Try First[dm] // InputForm and if there are quotes (") around the numbers, you've imported text instead of numbers. How did you export and import to CSV (Export and Import)? Please show the code you used. — creidhne, May 20 '21 at 22:24
@Drod Thank you for your suggestion. Yes Dimension returns {} for dm and {30,2} for dmHalo. — asorlik, May 21 '21 at 19:11
@creidhne : halo38 = Import["row_data/avgTables_halo38.csv"]; dm = halo38[[5]][[2]] I used this. I now realize dm is imported as a string and not as a list.
Is there a way to import as a List? I've tried importing with the keywords "Data", "Table", and "List" and nothing has worked.

Thank you all! — asorlik, May 21 '21 at 19:17
I also used the following code to export:
Export[{"row_data/avgTables_halo1.csv"}, halo1]

Without adding "List" to the specifications so I'm not sure if I an recover the format in an easy way. — asorlik, May 21 '21 at 19:20
I just tested exporting the data with Export["row_data/testHalo.csv", dmHalo, "Data"] and it imported as a List. My problem now is that I have a large set of data that would take a long time to regenerate in the right format. Is there a way to salvage all of the data I have saved as strings? — asorlik, May 21 '21 at 19:34
EDIT: It seems that the "Data" specifier did not work because when I access the nested list it is still a String and not a List. I have tried using .dat and .txt and none work. It seems to be a problem with the format of my lists themselves, since they are in the format: {{ 1 ,{ {}...{} , {}...{} }}}. I think this format isnt easily understood by the different formats... — asorlik, May 21 '21 at 20:04

score 1 · Answer 1 · answered May 24 '21 at 19:40

The nested lists are saved as strings so ToExpression has to be mapped.

data = {{1, {{2, 3, 4}, {5, 6, 7, {8, 9}}}, 10, {11, 12}}};
export = ExportString[data, "CSV"];
import = ImportString[export, "CSV"];
importData = import // Map[ToExpression]
importData == data
(* True *)

score 0 · Answer 2 · answered May 24 '21 at 19:04

Summary of issue: dmHalo and dm in the original post were inner parts of a nested data structure in the format {{i, {{j,k,l},{m,n,o},{...}}}}. The problem is that the export formats--.csv, .dat, .txt.-- regardless of keyword--"Data","Table","List"--would convert the inner lists into strings in a way that is irreversible (at least from what I have tried i.e. using ToExpression on the resulting string).

SOLUTION: Using Put and Get (Mathematica: issues storing nested list on disk) has seemed to resolve the issue and I now can recover the list format of the inner nested lists. As seen in the images, the export process converts the inner list to a string, whereas with the Put and Get commands the list format is preserved.

Import from CSV formatting issue

2 Answers2