My Math Forum  

Go Back   My Math Forum > College Math Forum > Real Analysis

Real Analysis Real Analysis Math Forum

LinkBack Thread Tools Display Modes
August 19th, 2012, 02:07 AM   #1
Joined: Aug 2012

Posts: 1
Thanks: 0

Interpolation of larga data sets

This is my first post so I wish to say hello to everyone Hello!

Here is my problem. I have fairly large data sets of around 100-200k of natural numbers for which I wish to find a function, relation, series or a sequence (I don't really care). For the purposes of a discussion, let's say it is a series. The only thing this series has to be able to do is map element "k" (0...k...n) to a value from the dataset. I do not care what happens to the series after the final value or how it behaves before first element. Another issue is that errors are only acceptable at decimal point (ie. if index 5 gives value 110, then it is OK if it shows anything between 9.5 and 10.49 but nothing else). All values have to be mapped and no additional values can show up. Can you help me?

I made some research and I think Gauss's Interpolation Formula might be what I am looking for, but I am not sure. Also, I do know neural network could be trained to fit the data, but it is too CPU expensive since I would have to train many networks a day (which is why I am looking for alternative methods).
godric is offline  
August 31st, 2012, 05:53 AM   #2
Senior Member
Joined: Aug 2012

Posts: 229
Thanks: 3

Re: Interpolation of larga data sets

Hey godric.

You can interpolate between the points quite easily but the function you create will be very complicated and probably just as complicated as the information embedded in the actual points themselves.

Typically in modern mathematics, what happens is that you use what are known as projection based techniques.

Projection based techniques can take a signal of some sort (like your points) as well as a model of a function and it can project those points to the basis vectors corresponding to the function and then recombine the results to get an approximated version of the signal if the basis you are using is not of infinite-rank (just think of it as an approximation of the signal and not the absolute representation for simplicity).

So if you want to use a particular model like a bunch of wave functions, some nth degree polynomial or some other function, you can construct a basis on an interval, project your signal to that basis and then re-construct an approximation of your signal with respect to that basis by using the theory of Hilbert-spaces and the infinite-dimensional vector space theory concerning the results of projections onto L^(R).

If you are not going to use something like a neural network, you will need to decide the basis yourself: I would suggest that you choose a fourier series over the interval and narrow the band (i.e. keep as many lower frequencies as required).

I can run through this with you if you don't have much math experience, but the idea is that this is basically a way of reconstructing your signal using the standard signal processing idea of treating the input as a bunch of sines and cosines and extracting the co-effecients for each set of individual frequencies.
chiro is offline  

  My Math Forum > College Math Forum > Real Analysis

data, interpolation, larga, sets

Thread Tools
Display Modes

Similar Threads
Thread Thread Starter Forum Replies Last Post
Two data sets fitting Simon39 Abstract Algebra 4 November 29th, 2013 09:41 PM
Combining 2 data sets Shakarii Advanced Statistics 1 August 10th, 2012 02:32 AM
Math - Data Management problem involving sets and subsets. police1.police2 Algebra 1 December 16th, 2011 12:57 PM
Help on comparing differing sets of data carter22 Algebra 2 March 1st, 2010 05:49 PM
Comparing multiple data sets with different n values cboldon Advanced Statistics 2 April 7th, 2008 03:01 PM

Copyright © 2019 My Math Forum. All rights reserved.