A different kind of sample: Counting Sample

Counting Sample

A Counting Sample S is obtained using the following counting algorithm:

S = empty; while ( not EOF ) { t = read_next_tuple(); // Get next tuple from input if ( t ∈ S ) { increase count of the t value in S } else if ( random() < 1/T ) { add (t,1) to S } }

Example:



  Input stream:

        x  x  v  v  x  x  v  x  x  x  v  v  x  x  x  x  v  x  v  v  x  ...    
              |  |        |           |  |              |     |  |
         not picked       |           |  |              |     |  |
   (probability 1-1/T)    |           |  |              |     |  |
                          |           |  |              |     |  |
		      First one       |  |              |     |  |
		      picked          |  |              |     |  |
		      (probab 1/T)    |  |              |     |  |
                                      v  v              v     v  v
		         c=1         c=2 c=3	       c=4  c=5 c=6
                                     subsequent values added with probab = 1

Operation of the Counting Sample algorithm:

For each input value v, if v in not in S, we will add v with probability 1/T
But if v is already contained in S, we simply increase the corresponding count

Motivation:

If v is already contained in S, then we have created an entry in S to keep a count of v
It will not cost more memory space to keep an accurate count of v, so why not increase the corresponding count ?

NOTE:

The Counting Sampling algorithm does not produce a uniform random sample !
That is because the values in the counting sample were not selected using a fixed probability:

NOTE:

Obtaining a Consise Sample from a Counting Sample

Remember that our goal is:
A random sample obtained using the counting sample algorithm is not a uniform sample:
Although a Counting Sample is not a uniform sample from the input stream, it can be converted into a uniform sample (with a probability 1/T)

Algorithm to convert a Counting Sample into a Concise Sample:

S = Counting Sample for ( each entry (v,c) &isin S ) do { for (i = 1; i <= c-1; i++) { if ( random() < 1/T ) c = c - 1; } }

Claim:

The output of the above algorithm is a Concise Sample
More precisely:

Proof:

Consider how an entry (v, c) in a Counting Sample is obtained:

Observation:

The values v upto and including the first v value picked by the Counting Sample algorithm, were selected with probability 1/T
The remaining "c-1" values in the input stream were selected with probability 1

Therefore, to obtain a random sample with selection probability 1/T, we must subject the c-1 "unselected" value to the same random selection
This program fragment is doing the c-1 (random) selections:

Incremental maintenace of the Counting Sample

Recall the Counting Sample algorithm:

S = empty; while ( not EOF ) { t = read_next_tuple(); // Get next tuple from input if ( t ∈ S ) { increase count of the t value in S } else if ( random() < 1/T ) { add (t,1) to S } }

Problem of this algorithm:
Just like the Concise Sample algorithm, we can drop elements from the current Counting Sample to obtain a smaller Counting Sample
But, we must make sure that:

The Incremental maintenance of a Counting Sample:

T = 1; // Selection probability = 1/T // Initial selection probability S = empty; // S = the counting sample while ( not EOF ) { // --------------------------------------------- // Inserting values in counting sample // --------------------------------------------- if ( t ∈ S ) { increase count of the t value in S } else if ( random() < 1/T ) { add (t,1) to S } // --------------------------------------------------- // Reduce counting sample when size exceeds threshold // --------------------------------------------------- if ( size(S) > threshold ) { T' = α × T; // α > 1 // T' is the new sampling probability for ( each t ∈ S ) { // ---------------------------------- // Adjust first element in the count // ---------------------------------- if ( random() < T/T' ) continue; // success: select remaining // element with probab. = 1 else { t.count--; // fail: delete // --------------------------------------- // Adjust remaining elements in the count // --------------------------------------- while ( t.count > 0 ) { if ( random() < 1/T' ) break; // success; add remaining value // with probab. 1 else t.count--; // fail: delete } if ( t.count == 0 ) delete t from S } // else } // for T = T'; // New selection probability... } }

Theorem 5: Correctness of the Counting Sampling algorithm

Claim:

Proof:

The algorithm has two part:
- Adding a (new) element
- Removing elements

Clearly, the additive part of the algorithm will add a uncounted item from the input stream with current probability 1/T because of the code:

// --------------------------------------------- // Inserting values in counting sample // --------------------------------------------- if ( t ∈ S ) { increase count of the t value in S } else if ( random() < 1/T ) { add (t,1) to S }

(I say current because the selection probability is adjusted when the size gets too large)

And if the value is in S, the algorithm will increase its count.

In order to show that the algorithm maintains a counting sample with probability 1/T from the input stream, we must show:

The delete algorithm produces a counting sample from the input stream with sampling probability 1/T'
The remainder items from the input stream is also sampled probability 1/T'

Consider the deletion algorithm:

T' = α × T; // α > 1 // T' is the new sampling probability // ---------------------------------- // Adjust first element in the count // ---------------------------------- if ( random() < T/T' ) exit; // success: select remaining // elements with probab. = 1 else t.count--; // fail: delete // --------------------------------------- // Adjust remaining elements in the count // --------------------------------------- while ( t.count > 0 ) { if ( random() < 1/T' ) break; // success; add remaining value // with probab. 1 else t.count--; // fail: delete } if ( t.count == 0 ) delete t from S

Denote:

S = Counting Sample before deletion
S' = Counting Sample after deletion
(well, we don't really know that S' is really a Counting Sample yet :-))

We know that S is a Counting sample from the input stream with selection probability 1/T

We will show that:

S' is a Counting sample from the input stream with selection probability 1/T'
(With T' = α × T)

Overview of the delete operation:

First X picked with probab 1/T input stream: v1 X v3 v4 X v6 X ... X ... X ... | | | | | | | | | subsequent X are | | | | | | | | | selected with probab. 1/T v v v v v v v v v S: X X X X ... | | if success, the rest are | | selected with probab. 1 v v S': X? X? ...
Category of values: (1) (2) (3) (3) (3)

The proof again goes by considering 3 categories of values:

The values X in the input stream before the first selected value (they were not included in S):

A values X is included in S with probability 1/T

A value is included in S' with probability 1/T' (and 1/T' < 1/T)

Because if random() < 1/T, then random() < 1/T', therefore:

Question: did the algorithm introduce values that were not in S ?
Answer: No, the algorithm does not add new values to S

Good, on to the next category of values

The first value X in the input stream that was included in S

The first value X is included in S with probability 1/T
(A subsequent value X is included in S with probability 1)

A first value X must be included in S' with probability 1/T'

The code fragment to perform this adjustment is:

// ---------------------------------- // Adjust first element in the count // ---------------------------------- if ( random() < T/T' ) exit; // success; // exit ensure selecting remaining // elements with probab. = 1 else t.count--; // fail: delete

Question: was the first value X in S' selected with probability 1/T' ?
Answer: Yes, because that value is now selected into S with probability:

The values X after the first selected value in the input stream (these were included in S with probab. = 1)

If step 2 was successful, then the other value X is included in S' with probability = 1
If step 2 was unsuccessful, then the other value X is included in S' with probability = 1/T' until the first successful inclusion

The code fragment to perform this adjustment is:

// --------------------------------------- // Adjust remaining elements in the count // --------------------------------------- while ( t.count > 0 ) { if ( random() < 1/T' ) break; // success; add remaining value // with probab. 1 else t.count--; // fail: delete }

Questions:
- If step 2 was successful, will the other value X is included in S' with probability = 1 ?
  Answer: Yes, because if random() < T/T', the algorithm will exit and will not decrease t.count further.
- If step 2 was unsuccessful, will the other value X is included in S' with probability = 1/T' until the first successful inclusion ?
  Answer: Yes, according to the delete code fragment, each value is included with random() < 1/T', until and the first success further.

So after the deletion algorithm, the resulting sample is again a counting sample but obtained using the selection probability 1/T'
The algorithm then continues to sample with the new (adjusted) probability (resulting in a smaller sample size)
(Ingenious :-))