Glad to hear its definitely sediment.
If you use a gravel siphon now before the plant roots become too established, you can probably get it mostly clean.
2-3 inches thick would not be too thick, being that your planting live plants... sounds about right.

__________________