How to clean up invalid UTF8 characters
Ever since getting error on your Ruby on Rails application and saw many weird parameters on the logs? Or seeing many weird parameters send on its request header and don't know what to do ? I experienced through that problem too. The application I developed use UTF8 as the character encoding and I found all of those weird characters are invalid UTF8 character. Probably, someone tried to do an attack to that application by sending invalid characters to make the application raised an errors and exposed the security hole, who knows... I haven't investigated that far.
But one thing for sure, I must solve the problem by filtering those invalid UTF8 characters, before they reached any of the application's controllers. How did I do that ?
I started with searching some gems with such functionality. Among gems I found , which claimed to provide the functionality for cleaning up invalid UTF8 characters, I ended up to use rack-utf8_sanitizer. Just install that gem, follow the documentation and guides there , and you'll set. All of invalid UTF8 characters won't reach any of your application's controller. It will be filtered on the middleware.
Do you have any experience with other gems or plugins ? Let us know through the comment box below this post.