---

Better Algorithm for Uniform Coin Flip Sampling: Concise Sample

---

• Previously: naive Coin Flip Sampling algorithm
  • Uniform Coin Flip Random Sample algorithm:

    N = 0; // N = Number of items processed NOTE: algorithm will select an avg. of f*N tuples... while ( not EOF ) { t = read_next_tuple(); // Get next tuple from input // Select tuple if coin toss is success... // random() returns a random number in (0..1) if ( random() < f ) { Sample[N] = t; N = N+1; } }

  • Problems with this algorithm:

    The number of items selected is not pre-determined The number of items selected depends on how often the test random() < f succeeds, The representation is inefficient Example: Input stream: 1, 2, 3, 4, 5, 1, 3, 4, 1, 2, 3, ... Output: 1, 4, 4, 3 A better way is to represent repeated value more concisely

  ---

  ---

• Predicatable Sampling size
  • The paper New Sampling-Base Summary Statistics for Improving Approximate Query Answers presents two interesting Uniform Coin Flip sampling algoriithms that uses a fixed size memory space

  • A copy of the paper can be found here: click here

  • The algorithms in the paper are:

    Concise Sample algorithm Counting Sample algorithm

  ---

  ---

• Concise Sample
  • Concise Sample:

    A Concise Sample is a uniform (coin flip) sample of the data set such that: a value that appears once is stored as a singleton a value that appears more than once is stored as a (value, count) Example: Data Set: {1, 2, 3, 4, 1, 2, 3, 1, 2, 5} The Concise Sample: S = {(1,3), (2,3), (3,2), 4, 5}

    ---

  • Simplification:

    I find the way to represent singleton separately very annoying Instead, I will use a simpler representation: A value is always stored as: (value, count) This modification will increase the memory requirement slightly, but will simplify the discussion Example: Data Set: {1, 2, 3, 4, 1, 2, 3, 1, 2, 5} The Concise Sample: S = {(1,3), (2,3), (3,2), (4,1), (5,1)}

  ---

  ---

• Naive uniform (coin flip) sample using Concise Sample representation
  • We can re-write the naive coin-flip sampling algorithm using the Concise Sample representation

  • Naive Coin Flip Sampling using Concise Sample representation

    S = empty; while ( not EOF ) { t = read_next_tuple(); // Get next tuple from input // Select tuple if coin toss is success... // random() returns a random number in (0..1) if ( random() < f ) { if ( t ∈ S ) { increase count of the t value in S } else { add (t,1) to S } } }

    ---

  • The sample size of the re-written algorithm will still be random; depending on how often the test random() < f succeed...

  ---

  ---

• Producing a predicatable output sample size
  • The key to produce a predictable output sample size is:

    delete some entries from the output sample when the output sample grows too large

    Example: We want to keep the sample size <= 4

    Current Sample: S = { (1,3), (2,4), (3,2), (5,2) } Next input: 6 We reached the maximum size of 4 entries and must delete some entries from S before proceeding...

    ---

  • However, there is a condition that we must satisfy:

    After deleting some entry/entries from the Concise Sample S, the remaining entries in S must still be a uniform coin flip sample !!!

  ---

  ---

• Incremental maintenance of Concise Samples
  • Algorithm:

    T = 1; // 1/T = selection probability S = empty; // S = Concise Sample while ( not EOF ) { t = next input value; if ( random() < 1/T ) { if ( t ∈ S ) { increase count of the t value in S } else { add (t,1) to S } } /* ------------------------------------------- Deletion step: Adjust sample when it gets too large... ------------------------------------------- */ if ( size(S) > MaxSize ) { T' = α × T; // New selection probab for ( each sample t ∈ S ) do { for ( i = 1; i <= t.count; i++ ) { if ( random() < (1 - T/T') ) t.count--; } if ( t.count == 0 ) delete t from S; } T = T'; } }

    ---

  • The incremental maintenance algorithm is similar to the naive algorithm, except for the deletion step

  • In the deletion step, each entry in the Concise Sample is subject to removal with probability T/T'

  • Theorem 2 of the paper shows that the output of the incremental maintenance algorithm is a uniform random sample of the input stream using the selection probability 1/T:

  ---

  ---

• Theorem 2: Correctness of Concise Sample Algorithm
  • Claim:

    The output of the above algorithm is a concise sample where the elements of the input stream is sampled with the final probability 1/T

  • Proof:

    The algorithm has two part: Adding a (new) element Removing elements Clearly, the additive part of the algorithm will add the next item from the input stream with current probability 1/T because of the code: if ( random() < 1/T ) { // Add t to S if ( t ∈ S ) { increase count of the t value in S } else { add (t,1) to S } } (I say current because the selection probability is adjusted when the size gets too large) In order to show that the algorithm outputs a concise sample with selection probability 1/T, we must show that: The delete algorithm produces a random sampling from the input stream with sampling probability 1/T' The remainder items from the input stream is also sampled probability 1/T' Consider the deletion algorithm: T' = α × T; // New selection probab for ( each sample t ∈ S ) do { for ( i = 1; i <= t.count; i++ ) { if ( random() < (1 - T/T') ) t.count--; } if ( t.count == 0 ) delete t from S; } T = T'; Denote: S = Concise Sample before deletion S' = Concise Sample after deletion We know that S is a random sample from the input stream with selection probability 1/T We will show that: S' is a random sample from the input stream with selection probability 1/T' (With T' = α × T) Overview of the delete operation: input stream: v1 v2 v3 v4 v5 v6 v7 ... | | | | | | | | | | | | | | select with probab. 1/T v v v v v v v S: v2 v5 v7 ... | | | Removed with probab. (1 - T/T') | | | = select with probab. T/T' v v v S': v5 v7 ... The proof goes by considering 2 categories of values: Consider the values in the input stream that were not included in S: A value is included in S with probability 1/T A value is included in S' with probability 1/T' (and 1/T' < 1/T) Because if random() < 1/T, then random() < 1/T', therefore: A value that was not included in S must not be included in S' if S' is to be a random sample with probability 1/T' Question: did the algorithm introduce values that were not in S ? Answer: No, the algorithm does not add new values to S Good, on to the next category of values Consider the values in the input stream that were included in S and included in S' A value is included in S with probability 1/T A value is included in S' with probability 1/T' (and 1/T' < 1/T which is a stronger selection condition) Because S' is a random sample with probability 1/T', a value must be included in S' with a probability 1/T' Question: were the values in S' selected with probability 1/T' ? Answer: Yes, because a value is selected into S with probability: 1/T × T/T' = 1/T'

  • So after the deletion algorithm, the resulting sample is a concise sample obtained using the selection probability 1/T'

  • The algorithm then continues to sample with the new (adjusted) probability (resulting in a smaller sample size)

    (Ingenious :-))