My Math Forum Interpolation of larga data sets
 User Name Remember Me? Password

 Real Analysis Real Analysis Math Forum

 August 19th, 2012, 03:07 AM #1 Newbie   Joined: Aug 2012 Posts: 1 Thanks: 0 Interpolation of larga data sets This is my first post so I wish to say hello to everyone Hello! Here is my problem. I have fairly large data sets of around 100-200k of natural numbers for which I wish to find a function, relation, series or a sequence (I don't really care). For the purposes of a discussion, let's say it is a series. The only thing this series has to be able to do is map element "k" (0...k...n) to a value from the dataset. I do not care what happens to the series after the final value or how it behaves before first element. Another issue is that errors are only acceptable at decimal point (ie. if index 5 gives value 110, then it is OK if it shows anything between 9.5 and 10.49 but nothing else). All values have to be mapped and no additional values can show up. Can you help me? I made some research and I think Gauss's Interpolation Formula might be what I am looking for, but I am not sure. Also, I do know neural network could be trained to fit the data, but it is too CPU expensive since I would have to train many networks a day (which is why I am looking for alternative methods).
 August 31st, 2012, 06:53 AM #2 Senior Member   Joined: Aug 2012 Posts: 229 Thanks: 3 Re: Interpolation of larga data sets Hey godric. You can interpolate between the points quite easily but the function you create will be very complicated and probably just as complicated as the information embedded in the actual points themselves. Typically in modern mathematics, what happens is that you use what are known as projection based techniques. Projection based techniques can take a signal of some sort (like your points) as well as a model of a function and it can project those points to the basis vectors corresponding to the function and then recombine the results to get an approximated version of the signal if the basis you are using is not of infinite-rank (just think of it as an approximation of the signal and not the absolute representation for simplicity). So if you want to use a particular model like a bunch of wave functions, some nth degree polynomial or some other function, you can construct a basis on an interval, project your signal to that basis and then re-construct an approximation of your signal with respect to that basis by using the theory of Hilbert-spaces and the infinite-dimensional vector space theory concerning the results of projections onto L^(R). If you are not going to use something like a neural network, you will need to decide the basis yourself: I would suggest that you choose a fourier series over the interval and narrow the band (i.e. keep as many lower frequencies as required). I can run through this with you if you don't have much math experience, but the idea is that this is basically a way of reconstructing your signal using the standard signal processing idea of treating the input as a bunch of sines and cosines and extracting the co-effecients for each set of individual frequencies.

 Tags data, interpolation, larga, sets

 Thread Tools Display Modes Linear Mode

 Similar Threads Thread Thread Starter Forum Replies Last Post Simon39 Abstract Algebra 4 November 29th, 2013 10:41 PM Shakarii Advanced Statistics 1 August 10th, 2012 03:32 AM police1.police2 Algebra 1 December 16th, 2011 01:57 PM carter22 Algebra 2 March 1st, 2010 06:49 PM cboldon Advanced Statistics 2 April 7th, 2008 04:01 PM

 Contact - Home - Forums - Cryptocurrency Forum - Top