I changed the front sprocket. It seems that it was the cause of the annoying noise. It had play on the splines of the transmission shaft and in the front sprocket. I'll update more as soon as I have time to test it thoroughly, but I really think this was the problem!