5

This may be a silly question but if I have a long list (let's call it data)such as this:

https://pastebin.com/QCAKwZ2P

How can I make that list of a certain lenght?. In particular I want to subtract the entire list with a part of the same list but since they are of different length I cannot do it.

I would like to do something like this:

data-Select[data, 50 <= #[[1]] <= 105 &] but since data is of higher lenght than Select[data, 50 <= #[[1]] <= 105 &] it gives me an error. Is there a way around this?

Thank you.

EDIT:

I have been trying to do it with Interpolation like this:

dat1 = Select[data, 60 <= First[#] <= 140 &];
dat2 = Select[data, 10 <= First[#] <= 65 &];
fit1 = Interpolation[dat1];
fit2 = Interpolation[dat2];

fit1[#] - fit2[#] &;

but doing it with interpolation in this way is not working for me either perhaps because the interpolations are of different sizes as well?.

John
  • 1,611
  • 4
  • 14
  • I’m not sure I even follow the logic here. What do you expect to happen? If it was 1 part of the list, sure, then you just do a ConstantArray which is the length of the list you’re pulling from, but this seems like there will be a sequence that is found, and the operation of subtracting part of the list from the whole list honestly does not make any sense to me. Maybe I am misinterpreting, but how would the elements be distributed? This does not seem standard hence my requests for clarification. – CA Trevillian Jun 22 '20 at 02:05
  • @CATrevillian Thanks for your comment. I simply mean if there is a way to subtract the entire list with a part of the list. For example, I was trying to use something like data[[;; ;; 50]] to get essentially the same that you will get from data (taking every 50 points or so) and at the same time reducing the lenght of data resulting in a lenght of 58 in this case. I just hoping to find something similar in which I can reduce the lenght of data such as I can subtract some other part of the same data. I hope that makes more sense. – John Jun 22 '20 at 02:16
  • I was thinking that another approach could be perhaps fitting part of the data to an interpolation function and simply subtract data-part of data (from the interpolation). – John Jun 22 '20 at 02:19
  • If you want to perform addition operations (subtraction is adding the negated value) of multiple lists, they will need to be the same length. Say you have Range[10] and you want to subtract {7, 8, 9} from the entire list, what do you expect to happen—that is, what is your expected output? Please note that the word subtract is not the same as remove. – CA Trevillian Jun 22 '20 at 02:22
  • @CATrevillian yes, I understand that if they are different lenghts they simply cannot be subtracted. That's why my example with data[[;; ;; 50]]] (which reduces the lenght of data while more of less keeping the overall trend of the data) and also my example of perhaps fitting the entire data and then there should be a way to subtract a part of the fitting of data with the entire fitting of data, no? – John Jun 22 '20 at 02:29
  • Why is it that you want to do this? Please regard this as asking for clarification to potentially avoid an XY problem. – CA Trevillian Jun 22 '20 at 04:09
  • @CATrevillian I edited a little bit the questions to try to make it more clear and so that you can see what I have tried with Interpolation. I just want to do it to subtract one part of the data (the part belonging to dat1 above) to for instance dat2 (above too). – John Jun 22 '20 at 04:25
  • Are you trying to subtract something like a parasitic signal or something? The interpolation method may work if you then make lists of equal length. Possibly something with piecewise might be useful. I’m not sure here. I hope someone may give you an adequate answer! – CA Trevillian Jun 22 '20 at 05:00

4 Answers4

5

TimeSeries provides a fairly direct approach to this:

data = Import["https://pastebin.com/raw/QCAKwZ2P", "Package"];

dat1 = Select[data, 60 <= First[#] <= 140 &];
dat2 = Select[data, 10 <= First[#] <= 65 &];


ts2 = TimeSeries[dat2] ~TimeSeriesRescale~ {60, 140};


dat3 = {#, #2 - ts2[#]} & @@@ dat1;


ListPlot[{dat1, dat2, dat3}]

enter image description here

See also TimeSeriesResample.

Addendum

If I follow what you're asking for in the comments, try this:

ts2raw = TimeSeries[dat2];

datX = Array[{#, ts2raw@#} &, Length @ dat1, MinMax[First /@ dat2]];

Closely related but allowing for nonuniform sampling:

{t1, t2} = {dat1, dat2}[[All, All, 1]];
{m1, m2} = MinMax /@ {t1, t2};
ts2 = TimeSeries[dat2];

datY = {#, ts2@#} & /@ Rescale[t1, m1, m2];

Or:

{ts1, ts2} = TimeSeries /@ {dat1, dat2};
times = TimeSeriesRescale[ts1, MinMax @ ts2["Times"]]["Times"];

tsX = TimeSeriesResample[ts2, {times}]
Mr.Wizard
  • 271,378
  • 34
  • 587
  • 1,371
  • Thank you very much Mr. Wizard!!! – John Jun 22 '20 at 16:22
  • Mr Wizard. I think in your code there is a similar comment than what I did to C.E. Shouldn't dat2 and ts2 be the same when plotted together?. One should be simply the rescalation of the other but they are different – John Jun 22 '20 at 18:02
  • @John Certainly they shouldn't cover the same x-range; that's the whole point of rescaling? Or perhaps I misunderstand? What exactly are you seeing? – Mr.Wizard Jun 22 '20 at 18:56
  • Mr. Wizard, for instance if I plottem such as: ListPlot[{dat2, ts2, dat1}] then you can see that dat2 and ts2 are very different. What I wanted is for instance for dat2 and ts2 to be equal with the only different being that one is rescaled in a way that they cover the same range but one is every 10 points or so so that it has the same length that dat1 but still cover the same range – John Jun 22 '20 at 19:26
  • @John Please see the addendum in my answer; is datX what you are looking for? – Mr.Wizard Jun 22 '20 at 19:55
  • Mr Wizard! Yes, datX is exactly what I was looking for!. As you can see now datX is the same as dat2 but I can do dat1-datX which is something I was not able to do with dat2 by trying dat1-dat2 – John Jun 22 '20 at 20:02
  • @John Okay, glad we got that sorted out. :-) – Mr.Wizard Jun 22 '20 at 20:06
  • @John I added a couple of other methods to the addendum that you may wish to consider. – Mr.Wizard Jun 22 '20 at 20:25
  • Mr. Wizard, thank you. You are awesome! I have further edited the post to reflect one possible use of your code that I am trying but I am not getting the plot to have zero baseline. I would like to have that also for other people as a way to use your code to subtract a baseline. Am I using your code incorrectly the way I showed it? – John Jun 22 '20 at 21:01
  • Ah, you want to remove the background of a signal! – CA Trevillian Jun 22 '20 at 23:01
  • @CATrevillian Yes! Exactly!. Well, that's one application I have in mind. In fact: I put it as a question here: https://mathematica.stackexchange.com/questions/224518/remove-baseline-from-curve using Mr. Wizard's code – John Jun 22 '20 at 23:03
5

Here's how to get the same result as Mr. Wizard with your interpolation approach (perhaps there is a slight difference in the result, but the idea is the same):

dat1 = Select[data, 60 <= First[#] <= 140 &];
dat2 = Select[data, 10 <= First[#] <= 65 &];
interp1 = Interpolation[dat1];
interp2 = Interpolation[dat2];

rescaled = Quiet@Table[{0, interp2[t]}, {t, Most@Subdivide[10., 65., Length[dat1]]}];
ListLinePlot[{dat1, dat2, dat1 - rescaled}]

Output

C. E.
  • 70,533
  • 6
  • 140
  • 264
  • Aweome! Thank you C.E ! – John Jun 22 '20 at 16:21
  • I think the reason the code gives a slightly different result than Mr.Wizard is that in your code if you compare interp2 and rescaled they are different and they should be the same as one is simply the rescaled of the other, no? – John Jun 22 '20 at 17:57
  • To follow up the idea if I use for instance ListLinePlot[{interp2, rescaled, dat1}] you can see that dat2 and rescaled are very very different and in fact rescaled goes to about 1300 in the x-axis rather than to only from 10 to 73. – John Jun 22 '20 at 18:12
  • 1
    @John I made a small mistake, but it didn't impact the result a whole lot. Just a small ~1 unit offset along the x-axis. I fixed it now. – C. E. Jun 22 '20 at 19:05
  • C.E thank you for your effort and help. However, it seems that still if I plot ListLinePlot[{interp2, rescaled, dat1}] both interp2 and rescaled are very different. What I would like is for interp2 and rescaled to essentially superimposed with the only different being that rescaled will have data points such that it is the same lenght than dat1 (it will go from 10 to 65 with total steps of dat1) – John Jun 22 '20 at 19:40
  • @John I'm sorry, but I don't understand what you're saying. If you take interp2 and rescale it, then obviously it will not be the same as interp2. rescaled does have the same length as dat1. Does it have the same x values? No, because the x values are all set to 0 to allow for subtraction in the way that Mr. Wizard did it. But you can easily change that. – C. E. Jun 22 '20 at 19:54
  • C.E please see the last comment I made to Mr. Wizard. In his code datX is similar to what I am looking for. As you can see now datX is the same as dat2 but I can do dat1-datX. datX in your code would be the equivalent of having interp2 with the difference that interp2 does not have the same range as dat1 and hence I cannot do dat1-interp2 but with something similar to datX in your code I could do dat1-datX and get the same as I would if I could do dat1-interp2. I hope this is more clear now – John Jun 22 '20 at 20:06
  • 1
    @John Well then I understood it correctly, but as I said, it's really a very trivial change. I would rather see that you figured it out yourself so that you understand the solution. You only have to change a single character in my solution to get the equivalent of Mr. Wizard's datX. – C. E. Jun 22 '20 at 20:16
  • C.E yes! Thank you I was able to change it to have datX. You have helped me so much!!! (+3) – John Jun 22 '20 at 21:02
3
ts = TimeSeries @ data;

{window1, window2} = {{60, 140}, {10, 65}};

{ts1, ts2} = TimeSeriesWindow[ts, #] & /@ {window1, window2};

dif12 = ts1 - TimeSeriesRescale[ts2, window1];

ListLinePlot[{ ts, dif12}]

enter image description here

kglr
  • 394,356
  • 18
  • 477
  • 896
0

Use Select directly to get the list you want, changing the selection predicate from what you have to accomplish this. Replace < by > or vice versa.

PaulCommentary
  • 1,482
  • 7
  • 12
  • 1
    I don’t think that answers OP’s question. I believe they have asked a similar question wherein that method was given as an answer. Subtract=/=remove, at least with how I read this. – CA Trevillian Jun 22 '20 at 02:14
  • Possibly. I read subtraction as as a way to make the list shorter given original post. But perhaps not correct interpretation on my part. – PaulCommentary Jun 22 '20 at 03:21