4

Any literature that extensively discusses the ability to strongly inter-/extrapolate computationally on little empirical data?

This topic has fascinated me, but I find that it seems a bit novel.

Particulary, instead of gathering large amounts of empirical observations, is there something that could guide interpolating a lot of empirical result from very limited but "skillfully" selected or manipulated data?

mavavilj
  • 427
  • 2
  • 8
  • If you can cook up an analytical model you might get some mileage from small samples. – A rural reader Oct 25 '21 at 18:38
  • 2
    What does "strongly interpolate" mean? – Wolfgang Bangerth Oct 25 '21 at 19:09
  • 1
    Look up krigging and radial basis functions. You can also see https://bbanerjee.github.io/ParSim/assets/tech_reports/TabularInterp_Nov_2020.pdf for a concise discussion of the basic ideas. – Biswajit Banerjee Oct 25 '21 at 19:36
  • @WolfgangBangerth Interpolate or extrapolate to empirical observations that haven't been seen. But that are nevertheless accurate based on the model. More specifically, consider having a "strongly interpolating" $f$ for which $f(smallsample) \approx bigsample_{features}$. Without having observed all features of $bigsample$. – mavavilj Oct 26 '21 at 07:07
  • 5
    That sounds like you just want to estimate the parameters of a model. If you don't have a model, there is nothing you can do with small amounts of data. – Wolfgang Bangerth Oct 26 '21 at 11:50
  • @Aruralreader What do you mean by "analytical model"? – mavavilj Nov 26 '21 at 13:16
  • @WolfgangBangerth I'm trying to gauge the possibility of such paradigm, because wouldn't it be nice if one did not have to collect large amounts of data? Or run a large amount of experiments? But one could nail down what the "main patterns" are by making careful deductions on small samples and using intuition on theory. I.e. that analysis of methods and a small sample might approach the phenomena of large samples, w/o requiring large samples. So e.g. to study cancer you'd only need to study few great examples of cancer, rather than a very large set of small variations of it. – mavavilj Nov 26 '21 at 13:16
  • 1
    Extrapolation, for example predicting future states, based on sparse data tends to be dicey even if you have a model of your system, process, etc. – A rural reader Nov 26 '21 at 13:21
  • Or i.e. also that one could posit that some phenomena are "in main line" governed by the rules of a small but well-chosen set, while the larger set is mostly still that, but where there are also more "small outliers". – mavavilj Nov 26 '21 at 13:22
  • @Aruralreader It's not sparse, if this small set of study samples is "very well chosen". I.e. that it very well represents larger sets, even when the set itself is small. – mavavilj Nov 26 '21 at 13:22
  • 2
    Polynomial interpolation in the non-scalar case is usually rather ill-conditioned, getting worse with higher dimension. Or said another way, given a set of monomials, it is or at least was some 10 years ago a topic of research to find point-sets where interpolation using these monomials is well-conditioned. – Lutz Lehmann Nov 26 '21 at 13:47
  • @LutzLehmann I'm not talking about only mathematical sense of interpolation, but also information -sense. I.e. if it's possible to develop models from small samples, where a human is able to "parse the information between". In this sense such model would be "like a model on large samples", but where a human is still able to interpret outliers based on the simpler model. I.e. a mix of expert input and model prediction. – mavavilj Nov 26 '21 at 15:14
  • Yes, that is the idea of machine learning for classification and encoding. The result is an, approximately or exactly, piecewise linear function. If one meets the balance of minimizing the structure, making the resulting function more rigid, while still being flexible enough to adapt to the data, then the result will be a good interpolation on the convex hull of the data set and a little beyond. Extrapolation without a model does not work. Even with a model, like Newton or general gravity, extrapolation did not work on a galactic scale, so dark matter/energy had to be "invented". – Lutz Lehmann Nov 26 '21 at 15:51
  • @mavavilj Beside Machine learning approaches I would highly suggest to consider rather rational polynomial approximations (e.g. Pade) instead of non-rational polynomial approximations (e.g. Taylor series), especially if you are interested in extrapolating your data. https://youtu.be/szMaPkJEMrw – ConvexHull Nov 26 '21 at 16:32

0 Answers0