
Real Analysis Real Analysis Math Forum 
 LinkBack  Thread Tools  Display Modes 
August 19th, 2012, 03:07 AM  #1 
Newbie Joined: Aug 2012 Posts: 1 Thanks: 0  Interpolation of larga data sets
This is my first post so I wish to say hello to everyone Hello! Here is my problem. I have fairly large data sets of around 100200k of natural numbers for which I wish to find a function, relation, series or a sequence (I don't really care). For the purposes of a discussion, let's say it is a series. The only thing this series has to be able to do is map element "k" (0...k...n) to a value from the dataset. I do not care what happens to the series after the final value or how it behaves before first element. Another issue is that errors are only acceptable at decimal point (ie. if index 5 gives value 110, then it is OK if it shows anything between 9.5 and 10.49 but nothing else). All values have to be mapped and no additional values can show up. Can you help me? I made some research and I think Gauss's Interpolation Formula might be what I am looking for, but I am not sure. Also, I do know neural network could be trained to fit the data, but it is too CPU expensive since I would have to train many networks a day (which is why I am looking for alternative methods). 
August 31st, 2012, 06:53 AM  #2 
Senior Member Joined: Aug 2012 Posts: 229 Thanks: 3  Re: Interpolation of larga data sets
Hey godric. You can interpolate between the points quite easily but the function you create will be very complicated and probably just as complicated as the information embedded in the actual points themselves. Typically in modern mathematics, what happens is that you use what are known as projection based techniques. Projection based techniques can take a signal of some sort (like your points) as well as a model of a function and it can project those points to the basis vectors corresponding to the function and then recombine the results to get an approximated version of the signal if the basis you are using is not of infiniterank (just think of it as an approximation of the signal and not the absolute representation for simplicity). So if you want to use a particular model like a bunch of wave functions, some nth degree polynomial or some other function, you can construct a basis on an interval, project your signal to that basis and then reconstruct an approximation of your signal with respect to that basis by using the theory of Hilbertspaces and the infinitedimensional vector space theory concerning the results of projections onto L^(R). If you are not going to use something like a neural network, you will need to decide the basis yourself: I would suggest that you choose a fourier series over the interval and narrow the band (i.e. keep as many lower frequencies as required). I can run through this with you if you don't have much math experience, but the idea is that this is basically a way of reconstructing your signal using the standard signal processing idea of treating the input as a bunch of sines and cosines and extracting the coeffecients for each set of individual frequencies. 

Tags 
data, interpolation, larga, sets 
Thread Tools  
Display Modes  

Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
Two data sets fitting  Simon39  Abstract Algebra  4  November 29th, 2013 10:41 PM 
Combining 2 data sets  Shakarii  Advanced Statistics  1  August 10th, 2012 03:32 AM 
Math  Data Management problem involving sets and subsets.  police1.police2  Algebra  1  December 16th, 2011 01:57 PM 
Help on comparing differing sets of data  carter22  Algebra  2  March 1st, 2010 06:49 PM 
Comparing multiple data sets with different n values  cboldon  Advanced Statistics  2  April 7th, 2008 04:01 PM 