Sub hourly text file load is too large (>5 GB)

Need a better way of doing this, maybe dynamically querying sqlite file