Heat 2 tablespoons olive oil in a medium size dutch oven or large skillet set on medium-high heat.
Once hot, add the chicken, garlic, balsamic vinegar, smoked paprika, salt and pepper. Toss the chicken to coat and then cook until the chicken is browned all over and cooked through, about 5 minutes. Stir in the oregano and cook another minute. Remove the chicken from the pan and to a plate.
To the same pan, add another tablespoon of olive oil, the bell peppers and a pinch of salt + pepper. Sear the peppers until just beginning to caramelize on the edges, about 3-4 minutes. Add the orzo and another tablespoon of olive oil. Stir the pasta with the peppers on then pour in 2 1/2 cups water. Cook until almost all the water is absorbed and the orzo is creamy. Taste the orzo to make sure it is soft in the middle, if needed, add another 1/2 cup of water and continue cooking the orzo until soft. It is important to stir the orzo often, as it wants to stick to the bottom of the pan.
Once the orzo is cooked, slide the chicken and olives into the orzo and remove from the heat. Allow the chicken to warm through, about 3 minutes.
Serve the dish topped with crumbled feta cheese and a side of tomatoes in chili oil.