Let $A$ be a square-free abelian variety defined over a number field $K$. Let $S$ be a density one set of prime ideals $\p$ of $\mathcal{O}K$. A famous theorem of Faltings says that the Frobenius polynomials $P{A,\p}(x)$ for $\p\in S$ determine $A$ up to isogeny. We show that the prime factors of $|A(\FF_\p)|=P_{A,\p}(1)$ for $\p\in S$ also determine $A$ up to isogeny over an explicit finite extension of $K$. The proof relies on understanding the $\ell$-adic monodromy groups which come from the $\ell$-adic Galois representations of $A$, and the absolute Weyl group action on their weights. We also show that there exists an explicit integer $e\geq 1$ such that after replacing $K$ by a suitable finite extension, the Frobenius polynomials of $A$ at $\p$ must equal to the $e$-th power of a separable polynomial for a density one set of prime ideals $\p\subseteq\mathcal{O}_K$.