problem with Importing multiple files

Question

I need to import about 100000 "dat" files to my progrom. I want to do this using the last answer to this question:

Importing multiple files using a for-loop

I know that I should define a list of all the names of my files, and at the end, I Import all of them. But I have a problem with this. The names of my files obey a special pattern, like this:

"mutomu,dms41,x1,dms51,x2,q24,x3,q25,x4.dat"

Which x1-x4 are integers in this way:

{x1, 1, 21}, {x2, x1, 21}, {x3, 1, 15}, {x4, 1, 15}

. When I am defining the names of my files, I do it as:

allprobabilityfiles = {Flatten[Table[{{x1, x2, x3, x4},"mutomu,dms41,x1,dms51,x2,q24,x3,q25,x4.dat"}, {x1, 1, 21,1}, {x2, x1, 21, 1}, {x3, 1, 15, 1}, {x4, 1, 15, 1}], 3]};

but in this way, the result is:

....{{21, 21, 6, 5}, "mutomu,dms41,x1,dms51,x2,q24,x3,q25,x4.dat"}, 
{{21, 21, 6, 6}, "mutomu,dms41,x1,dms51,x2,q24,x3,q25,x4.dat"}, 
{{21, 21, 6, 7}, "mutomu,dms41,x1,dms51,x2,q24,x3,q25,x4.dat"}, ....

(x1-x4 don't change inside the file names in this table), but what I need is the following:

....{{21, 21, 6, 5}, "mutomu,dms41,21,dms51,21,q24,6,q25,5.dat"}, 
{{21, 21, 6, 6}, "mutomu,dms41,21,dms51,21,q24,5,q25,6.dat"}, 
{{21, 21, 6, 7}, "mutomu,dms41,21,dms51,21,q24,5,q25,7.dat"}, ....

How can I solve this problem?

You can also refer to this solution :D – xzczd Apr 08 '13 at 05:39 — xzczd, Apr 08 '13 at 05:39

score 2 · Accepted Answer · answered Apr 08 '13 at 10:06

StringForm is useful for embedding values into a control string:

filestring[vals__] := 
 ToString @ StringForm["mutomu,dms41,``,dms51,``,q24,``,q25,``.dat", vals]

filestring[10, 20, 30, 40]
(*  "mutomu,dms41,10,dms51,20,q24,30,q25,40.dat"  *)

You would just drop this function into your Table like so:

allprobabilityfiles = 
  Flatten[Table[{{x1, x2, x3, x4}, filestring[x1, x2, x3, x4]}, 
   {x1, 1, 21, 1}, {x2, x1, 21, 1}, {x3, 1, 15, 1}, {x4, 1, 15, 1}], 3];

Thanks for your answer. I liked it :)... – tenure track job seeker Apr 08 '13 at 10:40 — tenure track job seeker, Apr 08 '13 at 10:40

yulinlinyu · Answer 2 · 2013-04-08T10:11:27.730

You can do like this:

Table[{x1, "mutomu," <> ToString[x1] <> ",dms51.dat"}, {x1, 
  Range@4}]

(* {{1, "mutomu,1,dms51.dat"}, {2, "mutomu,2,dms51.dat"}, {3, "mutomu,3,dms51.dat"}, {4, "mutomu,4,dms51.dat"}} *)

It is very easy to implement in your cases.

Edit:

Ok, I'll fit the code above for your case:

allprobabilityfiles = 
  Flatten[Table[{{x1, x2, x3, x4}, 
     "mutomu,dms41," <> ToString[x1] <> ",dms51," <> ToString[x2] <> 
      ",q24," <> ToString[x3] <> ",q25," <> ToString[x4] <> 
      ".dat"}, {x1, 1, 21, 1}, {x2, 1, 21, 1}, {x3, 1, 15, 1}, {x4, 1,
      15, 1}], 3];
allprobabilityfiles[[1 ;; 5]]

(* {{{1, 1, 1, 1}, "mutomu,dms41,1,dms51,1,q24,1,q25,1.dat"}, {{1, 1, 1, 2}, "mutomu,dms41,1,dms51,1,q24,1,q25,2.dat"}, {{1, 1, 1, 3}, "mutomu,dms41,1,dms51,1,q24,1,q25,3.dat"}, {{1, 1, 1, 4}, "mutomu,dms41,1,dms51,1,q24,1,q25,4.dat"}, {{1, 1, 1, 5}, "mutomu,dms41,1,dms51,1,q24,1,q25,5.dat"}} *)

More elegantly:

fileName := {{##}, 
    "mutomu,dms41," <> ToString[#1] <> ",dms51," <> ToString[#2] <> 
     ",q24," <> ToString[#3] <> ",q25," <> ToString[#4] <> ".dat"} &;
Flatten[Array[fileName, {21, 21, 15, 15}], 3]

Thanks for your answer, but this isn't what I am looking for. "mutomu,dms41,21,dms51,21,q24,6,q25,5.dat" (for example) is the name of my file, and at the end I should have something like that! — tenure track job seeker, Apr 08 '13 at 05:23
Thanks again. I just have one other question. I can see that <> must be available before and after ToString to put everything in the correct form. However, what is <> really doing? — tenure track job seeker, Apr 08 '13 at 06:13
StringJoin. I think you can type ?<> in Mathematica to get the information. — yulinlinyu, Apr 08 '13 at 09:35

score 1 · Answer 3 · answered Apr 08 '13 at 09:16

1

You may be able to use FileNames to select the files you want to import.

data=Import/@FileNames["/mydirpath/mutomu*dat"];

Or it may prove easier to move the data files to their own directory where you can use:

data=Import/@FileNames["/mydatadirpath/*"];

answered Apr 08 '13 at 09:16

image_doctor

10,234
23
40

Thanks for your answer. I knew this way, but I was avoiding to use it. I need to do it exactly as yulinlinyu's answer. – tenure track job seeker Apr 08 '13 at 09:28
I like this answer. – yulinlinyu Apr 08 '13 at 09:36
@Zahra Thank you, I'm glad you found the right solution to your problem :) – image_doctor Apr 08 '13 at 12:50

problem with Importing multiple files

3 Answers3