The current implementation of Kinesis sources will read in the entire stream upon connection. We should give users the option to create a source that starts at a specific:
Options are outlined more fully here: https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetShardIterator.html
A quick thing to think out here: if the stream contains changes then it might be a mis-feature to start only part of the way through the stream. On the other hand, if the stream is just lines of text that we are either appending or windowing or something, then no real harm.
Just a heads up that one prospect gave me the valuable feedback that this feature is super important because _oftentimes the start of the stream is full of bad dev data_ and things only iron out (in terms of schema, columns being present, etc.) after some point in time. They know the point in time, but they do need the knob to _throw away data_ before that point in time. This was in reference to their Kafka installation (related to #1210 which is the same issue for Kafka), but the general point applies to Kinesis as well.
(FYI the Kafka version of this is a 0.3 priority due to a prospect ask, but Kinesis prioritization should follow Kinesis prospect requests.)
@frankmcsherry opened a general musing ticket for this: https://github.com/MaterializeInc/materialize/issues/2862
Just to clarify: #2862 is about the ability to fast-forward the updates in a stream, rather than starting much of the way through a stream. They might be related, but they could also be very different. The default implementation of #2862 would be to read everything in a stream and just advance the timestamps found there, rather than skipping any of the stream.