Advances in quantum computing present a significant threat to cryptographic algorithms like RSA and ECC as quantum computers' ability to solve complex problems in polynomial time. In response to this, the National Institute of Standards and Technology (NIST) has started to design new cryptographic algorithms for the "post" quantum era and recently finalized the selection of post-quantum cryptographic (PQC) algorithms for standardization. While there have been some studies of integrating PQC into the TLS protocol for key establishment and signature generation, limited research exists on profiling these newly standardized algorithms in computational resource-constrained embedded systems for IoT usage. In this work, we first integrated NIST's recently standardized PQC algorithms into both TLS servers and clients built upon embedded systems. e.g., Raspberry Pi system with Arm Cortex-A72 64-bit SoC. With our experimental system, We benchmarked the performance of PQC algorithms and compared them with currently used classical algorithms. The Kyber, PQC Key Encapsulation Mechanism (KEM) algorithm, demonstrated a performance boost of up to 2.55x compared to the traditional ECDH algorithm. PQC signature algorithms (Dilithium, FALCON, and SPHINCS+) achieved 6.82x, 40.6%, and 9.3% performance of RSA for the signing operation and 27.5%, 1.46x, and 0.04% in terms of verification operation respectively. Additionally, we evaluate the PQC-TLS performance on the embedded SOC, revealing that handshake latency ranges (∼10-91% for Level 5) compared to non-PQC schemes. Furthermore, we are the first to evaluate the PQC-enabled QUIC protocol on an embedded system, aiming to optimize the performance overhead by leveraging QUIC's quick connection establishment trait as a UDP-only protocol.