1

I have a long table of data (the number of rows 795,000) and I need to create a code in Mathematica that can calculate the minimum value for every 5000 rows and then I need to know the positions of these minima.Is there any way to do this?

Ghady
  • 305
  • 1
  • 10

2 Answers2

3

First, you split yuor big data into chunks of size 5000 rows you will have a matrix like this:

 chunkOfMatrix = Table[RandomReal[], {i, 1, 5000}, {j, 1, 100}];

Then applying this will give the Min and its position in each row:

{Min[#], Position[#, Min[#]]} & /@ chunkOfMatrix

Hope this will help

David Baghdasaryan
  • 755
  • 4
  • 12
  • Thank you for your answer, but please can you explain more if my data is already as a matrix and its name is data, how can I use your command? – Ghady May 12 '17 at 14:27
  • Welcome, so if your data is smth. like this : data = Table[RandomReal[], {i, 1, 150000}, {j, 1, 100}]; and you can partition this data into chunks of 5000 rows using Partition function: dataChunks = Partition[data, 5000]; and then use the code above for every chunk. For example for the first one would be {Min[#], Position[#, Min[#]]} & /@ dataChunks[[1]]. Did I get you right? – David Baghdasaryan May 12 '17 at 15:07
  • I tried your way, but I didn't get what I need. For more details my data is below – Ghady May 12 '17 at 17:56
  • Sorry, my data is huge! – Ghady May 12 '17 at 18:05
  • Ohh, in this way you maybe need to write your data to a file and read it by portions. – David Baghdasaryan May 12 '17 at 19:54
  • I appreciate your help – Ghady May 12 '17 at 22:27
1

If you are looking for a running minima, then the answer already exists here. If you need it by chunks, then

Some data as an example (you use your own)

data = RandomReal[1, 795000];

The function

MinPositionAndValueByChunks[data_List, chunks_Integer] := MapIndexed[
  {First@FirstPosition[#1, Min[#1]] + chunks (First[#2] - 1), 
    Min[#1]} &,
  Partition[data, chunks]
  ]

Example

Short@MinPositionAndValueByChunks[data, 5000]

{{2042,0.0000435445},{6324,0.000265437},<<155>>,{787796,0.000635954},{793848,0.000139004}}
rhermans
  • 36,518
  • 4
  • 57
  • 149
  • I tried to use your command, but my data has specific numbers. It is the result of another code that computes the differential cross section, so do you think I can use RandomReal.? I should get stability minima. – Ghady May 16 '17 at 00:27
  • 1
    @Ghady You must be kidding, the random data I offer is just for testing. For the benefit of the people that do not have access to your data, as you did not share it. – rhermans May 18 '17 at 08:12