You can choose if you want your device to vibrate with each keystroke by following the instructions below:
1. Go to Settings, followed by Language & Input.
2. From there, select Android Keyboard/Google Keyboard, whichever you are using
3. From there, go to the Preferences section, and you can toggle the "Vibrate On Key Press" option to your desired choice.