let x be the sides of one polygon
y be the sides of the other polygon
x + y = 13
y = 13 - x
then the formula for the number of polygons is n(n - 3) / 2
x(x - 3) / 2 + y(y - 3) / 2 = 25
x(x - 3) / 2 + ( 13 - x) ( 13 - x - 3) / 2 = 25
x( x - 3 ) / 2 + ( 13 - x) ( 10 - x) / 2 = 25
solve for x
x = 8
y = 13 - 8 = 5
