Detect repeated business keys whose non-key values disagree and classify them for controlled resolution.
Caso práctico progresivo para transformar datos municipales sucios en información confiable, analizable y útil para decisiones.
Detect repeated business keys whose non-key values disagree and classify them for controlled resolution.
Este caso continúa el flujo real de trabajo con solicitudes de servicios municipales y documenta cada transformación de forma trazable.
Results can vary because the source dataset was generated synthetically and randomly. Run the code to obtain the exact values from your local file.
Los resultados pueden variar porque el dataset de origen fue generado de forma sintética y aleatoria. Ejecuta el código para obtener los valores exactos de tu archivo local.
import pandas as pd
from pathlib import Path
base = Path.home() / "Documents" / "Pandas_Use_Cases"
input_file = base / "exact_duplicates_removed_city_service_requests.csv"
output_file = base / "request_id_conflicts_resolved.csv"
conflict_file = base / "request_id_conflicts_audit.csv"
df = pd.read_csv(input_file)
mask = df["Request_ID"].duplicated(keep=False)
conflicts = df.loc[mask].sort_values("Request_ID")
print(f"Rows with repeated Request_ID: {len(conflicts):,}")
print(f"Repeated ID values: {conflicts['Request_ID'].nunique():,}")All records sharing a repeated business key are isolated.
comparison_columns = [
"Service_Type", "Department", "Status",
"Priority", "Resolution_Days", "Satisfaction_Score"
]
disagreement = (
conflicts.groupby("Request_ID")[comparison_columns]
.nunique(dropna=False)
)
disagreement["Has_Conflict"] = disagreement.gt(1).any(axis=1)
print(disagreement["Has_Conflict"].value_counts())Repeated IDs are classified as identical-key repeats or true content conflicts.
conflict_ids = disagreement.index[disagreement["Has_Conflict"]]
df["Request_ID_Conflict"] = df["Request_ID"].isin(conflict_ids)
print(df["Request_ID_Conflict"].value_counts())Each affected row receives a transparent conflict indicator.
# Example rule:
# Keep the most recently created record for each Request_ID.
df["Created_Date"] = pd.to_datetime(df["Created_Date"], errors="coerce")
resolved = (
df.sort_values(["Request_ID", "Created_Date"])
.drop_duplicates(subset="Request_ID", keep="last")
.copy()
)
print(f"Rows after key resolution: {len(resolved):,}")One explicit rule resolves the key conflict; organizations may substitute an approved rule.
conflicts.to_csv(conflict_file, index=False, encoding="utf-8-sig")
resolved.to_csv(output_file, index=False, encoding="utf-8-sig")
print(f"Resolved file: {output_file}")
print(f"Conflict audit: {conflict_file}")The resolved dataset and complete audit evidence are both retained.
The output of this module becomes the controlled input for the next stage of the 20-case Pandas workflow.
La salida de este módulo se convierte en la entrada controlada para la siguiente etapa del flujo de 20 casos de Pandas.
Explore additional resources connected to local training, practical skills, employment opportunities, and community growth.
Explora recursos adicionales relacionados con capacitación local, habilidades prácticas, oportunidades de empleo y crecimiento comunitario.
Community-focused resources for trades, skills, local opportunities, and practical career development.
Recursos comunitarios enfocados en oficios, habilidades, oportunidades locales y desarrollo profesional práctico.
hialeahoficios.comMiami-oriented resources for practical learning, employment pathways, and workforce development.
Recursos orientados a Miami para aprendizaje práctico, rutas de empleo y desarrollo de la fuerza laboral.
miamioficios.com