Can I have roles instead of users as a part of a group in an assignment rule?

Yes, you can add roles to a group. This is how it will work.

Create a master group that is made of multiple groups containing roles and users as members.  When you create an assignment rule for the master group, then all members (roles) in the group are considered for the assignment. 

For example, if Group 2 and Role 1 are Group 1’s members, then the members of Group 2 (P, Q, R) and Role 1 (X, Y, Z) become members of Group 1 (P, Q, R, X, Y, Z).

To know more about Assignment Rules, click here.