PublicDataReader

repository·main·Indexed 20 days ago

https://github.com/wooiljeong/publicdatareader

An open-source Python library that automates querying and organizing public data from sources including the Korean Public Data Portal, KOSIS, FRED, ECOS (Bank of Korea), Seoul Open Data Plaza, V-World, and KB Land. It simplifies data collection by handling API requests, authentication, and data parsing for real estate, economic, business, and geographic datasets.

Tokens
93.8K
Snippets
214
Records
259
Agent score
70%

What's inside PublicDataReader

  1. Overview of PublicDataReader capabilities

    main

    PublicDataReader is an open-source Python library designed to automate the retrieval of public data. It supports various Open API services such as the Public Data Portal (data.go.kr) and the Korean Statistical Information Service (KOSIS).

    Key features include:

    • Automated API Handling: Automatically manages API specifications, request construction, and data parsing.
    • Authentication Support: Handles data retrieval both with and without API authentication keys.
    • Data Processing: Automates the collection and processing of data that is publicly available on the web, simplifying the code required for data analysis.
  2. Explore the PublicDataReader package structure

    main

    The PublicDataReader package is organized into several subpackages and modules designed to interface with different public data sources. Developers can use specific subpackages depending on the data provider they need to access:

    • PublicDataReader.PublicDataPortal: For data from the Open Data Portal (data.go.kr).
    • PublicDataReader.Seoul: For data from the Seoul Open Data Plaza.
    • PublicDataReader.kosis: For data from the Korean Statistical Information Service (KOSIS).
    • PublicDataReader.config: For managing package configurations.
    • PublicDataReader.utils: For utility functions used across the package.
    • PublicDataReader.data: The core data module containing primary data retrieval logic.
  3. Use the Kamco Onbid Code Lookup Service

    main

    The Onbid Code Lookup Service (온비드코드조회서비스) allows you to retrieve hierarchical category codes and administrative area codes (Sido, Sigungu, Eupmyeondong, etc.) used by Kamco.

    All requests use the Kamco class and the get_data method. The service parameter must be set to '온비드코드'.

    from PublicDataReader import Kamco
    
    service_key = "YOUR_SERVICE_KEY"
    api = Kamco(service_key)
    
    # Example: Get top-level category codes
    df = api.get_data(
        service='온비드코드',
        function='용도상위코드'
    )
  4. Query Weekly Apartment Trend Statistics

    main

    Access weekly apartment market trends using the 주간아파트 service name. Available categories include 연령별거래유형별, 규모별거래유형별, 전세수급, 매매수급, and 지역별거래유형별.

    Note: You must have an active Open API key from the Korea Real Estate Board Open API page to use this service.

    service_name = "주간아파트"
    
    # List available categories
    category_list = list(api.meta_dict[service_name].keys())
    print(f"{service_name} 카테고리 목록")
    print(", ".join(category_list))
    
    # Fetch data for a specific category (e.g., '연령별거래유형별')
    category_name = "연령별거래유형별"
    df = api.get_data(
        service_name, 
        category_name,
        verbose=True,
    )
    print(df.tail(1))
  5. Query Officetel Price Trend Statistics

    main

    Access officetel price trends using the 오피스텔 service name. Categories include 지역별가격지수, 규모별가격지수, 지역별가격, 규모별가격, 지역별전세가격비율, 규모별전세가격비율, 지역별월세보증금비율, and 규모별월세보증금비율.

    Note: You must have an active Open API key from the Korea Real Estate Board Open API page to use this service.

    service_name = "오피스텔"
    
    # List available categories
    category_list = list(api.meta_dict[service_name].keys())
    print(f"{service_name} 카테고리 목록")
    print(", ".join(category_list))
    
    # Fetch data for a specific category (e.g., '지역별가격지수')
    category_name = "지역별가격지수"
    df = api.get_data(
        service_name, 
        category_name,
        verbose=True,
    )
    print(df.tail(1))
  6. Obtain an API key for Land Ownership Information

    main

    To use the Land Ownership Information Open API provided by the Ministry of Land, Infrastructure and Transport, you must obtain an authentication key from the Public Data Portal (data.go.kr).

    1. Register for an account on the Public Data Portal.
    2. Apply for the specific Open API service you wish to use.
    3. Wait for approval. Most API keys are available for use within 1-2 hours of application, though some services may require more than 24 hours for the authentication process to complete.
  7. Query Commercial Real Estate Lease Trend Statistics

    main

    Access commercial real estate lease trends using the 상업용임대 service name. Categories include 지역별전환율, 지역별임대가격지수, 지역별분기소득수익률, 지역별분기자본수익률, 지역별분기투자수익률, 지역별기타수입구성비, 순영업소득, 지역별영업경비구성비, 지역별임대수입구성비, 지역별임대료, 층별임대료, 층별효용비율, 지역별공실률, 지역별연간소득수익률, 지역별연간자본수익률, and 지역별연간투자수익률.

    Note: You must have an active Open API key from the Korea Real Estate Board Open API page to use this service.

    service_name = "상업용임대"
    
    # List available categories
    category_list = list(api.meta_dict[service_name].keys())
    print(f"{service_name} 카테고리 목록")
    print(", ".join(category_list))
    
    # Fetch data for a specific category (e.g., '지역별전환율')
    category_name = "지역별전환율"
    df = api.get_data(
        service_name, 
        category_name,
        verbose=True,
    )
    print(df.tail(1))
  8. Obtain API credentials for Housing License Information

    main

    To use the Ministry of Land, Infrastructure and Transport (MOLIT) Housing License Information services, you must obtain an authentication key from the Public Data Portal.

    1. Register as a member on the Public Data Portal.
    2. Apply for the '건축HUB_주택인허가정보 서비스' (Architecture HUB Housing License Information Service).
    3. Wait for approval. Typically, the key is available within 1-2 hours, but some APIs may take more than 24 hours to process.
  9. Retrieve PNU codes from Building Ledger and LandForestLedger

    main

    If you only have building information, you can derive PNU codes to query the LandForestLedger.

    1. Use pdr.code_bdong() to find the 법정동코드 (Legal Dong code) for a specific area.
    2. Use BuildingLedger.get_data() to retrieve building details.
    3. Construct the PNU code by combining: 시군구코드 (Sigungu code) + 법정동코드 (Legal Dong code) + 필지구분코드 (Parcel division code) + (Main number, zero-padded to 4 digits) + (Sub-number, zero-padded to 4 digits).

    Note: The 필지구분코드 (Parcel division code) needs to be mapped from the building ledger's 대지구분코드 (Land division code). Typically, '1' maps to '2' and '2' maps to '3' in the context of this transformation logic.

    import PublicDataReader as pdr
    from PublicDataReader import BuildingLedger, LandForestLedger
    
    service_key = "YOUR_SERVICE_KEY"
    
    # 1. Get Legal Dong codes
    code_bdong = pdr.code_bdong()
    # Filter for specific area (e.g., Seocho-gu, Seoul)
    area_codes = code_bdong[(code_bdong['시도명']=='서울특별시') & (code_bdong['시군구명']=='서초구')]
    
    # 2. Get Building Ledger data
    bl = BuildingLedger(service_key)
    buildings = bl.get_data("총괄표제부", sigungu_code="11650", bdong_code="10300")
    
    # 3. Transform and generate PNU
    transform_dict = {"0": "1", "1": "2", "2": "3"}
    buildings['필지구분코드'] = buildings['대지구분코드'].replace(transform_dict)
    buildings['PNU'] = (buildings['시군구코드'] + 
                        buildings['법정동코드'] + 
                        buildings['필지구분코드'] + 
                        buildings['번'].str.zfill(4) + 
                        buildings['지'].str.zfill(4))
    
    # 4. Use PNU to query LandForestLedger
    api = LandForestLedger(service_key)
    for pnu_code in buildings['PNU']:
        land_data = api.get_data(pnu_code=pnu_code)
        print(land_data)