its absolutely not worth sticking with enchanting...and alchemy is way better even without the speed pot recipe. reasons:
1. like everyone said, you can make oils really easy from a stockpiled stacks of dust (which you can get just from vendors).
2. Having Alchemy when you pop a Wisdom Elixir gives you +11 intel instead of the base +6...so with alchemy you get the full effect of both the oil and the elixir.
3. the oil stays on the weapon when you die, so you get the full 1hr duration. all other buffs fall off at death so you're going to burn through WAY more elixirs than oils.
After you stockpile all your enchants, Enchanting really shouldn't be more than something you powerlevel real quick to stock up on oils. You can alternate between herbalism and enchanting when you need the oils. Healers imo (and most casters) should all go with Alchemy and keep it.