To meet the increasing demand for low-latency processing in mobile and IoT devices, edge computing offloads user application tasks to nearby cloudlets (edge servers). Modern applications comprise multiple interdependent tasks with diverse service requirements that must be preloaded on cloudlets. This creates a coupled optimization challenge: jointly deciding task offloading locations and service caching across cloudlets. Inter-task dependencies further complicate the problem by introducing additional communication delays through data transfers and connectivity between cloudlets. Unlike existing studies that rely on a centralized controller with global knowledge, this work introduces a distributed framework for dependency-aware task offloading and service caching. This decentralized design reduces communication overhead and improves scalability. In the proposed framework, each cloudlet observes user task arrivals and network conditions, then applies a deep reinforcement learning (DRL) model enhanced with guided action shaping to determine task-offloading decisions. In parallel, service caching is performed independently by each cloudlet based on its observed service-demand statistics. Simulation results demonstrate that the proposed algorithm outperforms existing benchmarks for dependent task offloading and service caching. Moreover, it exhibits strong adaptability to dynamic environments, enabling more efficient and scalable edge computing.