4

I'm trying to import CSV file with the following format:

20150803 00:00:01.649,1.09675,1.09678

using this code:

Import[filename, 
  "DateStringFormat" -> {"Year", "Month", "Day", " ", "Hour", ":", 
    "Minute", ":", "Second", ".", "Millisecond"}]

But this does not import Time part - it only imports the Date:

{{DateObject[{2015, 8, 4}], 1.09675, 1.09682}, 
 {DateObject[{2015, 8, 3}], 1.09675, 1.09678},
 ...

How can I import all of the information from my file?

J. M.'s missing motivation
  • 124,525
  • 11
  • 401
  • 574
Slava Bogdanov
  • 43
  • 4

3 Answers3

2

Import does not support the time portion of DateStringFormat. (This is a deficiency in the docs if not a bug ) If you know the specific column(s) where your date appears you can map datelist like this:

MapAt[DateList[{#, {"Year", "Month", "Day", " ", "Hour", ":", 
 "Minute", ":", "Second", ".", "Millisecond"}}] & , 
      Import[filename.csv], {All, 1}]

In case you do not know where the strings might occur you can do this:

Map[Quiet@
  Check[DateList[{#, {"Year", "Month", "Day", " ", "Hour", ":", 
    "Minute", ":", "Second", ".", "Millisecond"}}], #] & , 
       Import[filename.csv], {-1}]

I would guess this will really kill performance on a large file though.

george2079
  • 38,913
  • 1
  • 43
  • 110
0
in = StringToStream[StringJoin[
    "20150803 00:00:01.649,1.09675,1.09678\r\n", 
    "20170930 23:59:59.649,1.19675,1.29678\r\n"]]

file = (StringSplit[#1, ","] & ) /@ 
   StringSplit[ReadString[in], RegularExpression[
     "[\r\n]+"]]

Close[in]

getNumber = Interpreter["Number"]

data = ({FromDateString[#1[[1]]], 
     getNumber[#1[[2]]], getNumber[
      #1[[3]]]} & ) /@ file

InputStream[Name: String
Unique ID: 3

]
{{20150803 00:00:01.649,1.09675,1.09678},{20170930 23:59:59.649,1.19675,1.29678}}
String
Interpreter[Number]
{{Mon 3 Aug 2015 00:00:01GMT-4,1.09675,1.09678},{Sat 30 Sep 2017 23:59:59GMT-4,1.19675,1.29678}}

The first cell would be replaced normally by an OpenRead to the CSV file.

0

I found solution at this answer:

And also improved loading speed of my 250Mb CSV file

Slava Bogdanov
  • 43
  • 4