Jupyter Notebook等の前段セルで認証情報が定義されている前提に基づき、ヘッダー情報を統合し、認証部分をコメントアウトした探索コードです。このセルを実行することで、README.first.txtの全文とmonthly_G直下の年号ディレクトリ一覧が出力されます。
=== Contents of README.first.txt ===
README.first (GSMaP_MVK & GSMaP_Gauge Ver.8)
********************************************************************************
* Documents
********************************************************************************
Readme (this file): /standard/v8/README.first.txt
Version up history: /standard/v8/GSMaP_MVK_RNL_HISTORY.txt
Caveat for algorithm: /standard/v8/doc/CaveatForUseOfGPM-GSMaP_Ver.8.pdf
Data Format Document: /standard/v8/doc/DataFormatDescription_MVK_RNL_v8.0000.pdf
User's Guide: /standard/v8/doc/USERS_GUIDE_MVK_RNL_v8.0000.pdf
********************************************************************************
* Archive Period and Products
********************************************************************************
Since January 1, 1998
Global Satellite Mapping of Precipitation Microwave-IR Combined Product (GSMaP_MVK)
Gauge-calibrated Rainfall Product (GSMaP_Gauge)
********************************************************************************
* Data Directory (ver.8) and File Naming Rule
********************************************************************************
Hourly Rain Rate data;
Directory: /standard/v8/hourly/YYYY/MM/DD/
File Name: gsmap_mvk.YYYYMMDD.HHNN.vP.RSKI.J.dat.gz (GSMaP_MVK)
Hourly Gauge-calibrated Rain Rate data;
File Name: gsmap_gauge.YYYYMMDD.HHNN.vP.RSKI.J.dat.gz (GSMaP_Gauge)
Satellite Information Flag;
Directory: /standard/v8/sateinfo/YYYY/MM/DD/
File Name: gsmap_mvk.YYYYMMDD.HHNN.vP.RSKI.J.sateinfo.dat.gz (GSMaP_MVK)
Observation Time Flag;
Directory: /standard/v8/timeinfo/YYYY/MM/DD/
File Name: gsmap_mvk.YYYYMMDD.HHNN.vP.RSKI.J.timeinfo.dat.gz (GSMaP_MVK)
Reliability Flag;
Directory: /standard/v8/reliability/YYYY/MM/DD/
File Name: gsmap_mvk.YYYYMMDD.HHNN.vP.RSKI.J.reliability.dat.gz (GSMaP_MVK)
Hourly Rain Rate & major flag in NetCDF;
Directory: /standard/v8/netcdf/YYYY/MM/DD/
File Name: gsmap_mvk.YYYYMMDD.HHNN.vP.RSKI.J.nc
Hourly Rain Rate & major flag in HDF;
Directory: /HDF/standard/v8/Hourly/YYYY/MM/DD/
File Name: GPMMRG_MAP_YYMMDDHHNN_H_L3S_MCH_VVV.h5
Daily Averaged Rain Data (00Z-23Z averaged);
Directory: /standard/v8/daily/00Z-23Z/YYYYMM/
File Name: gsmap_mvk.YYYYMMDD.0.1d.daily.00Z-23Z.vP.RSKI.J.dat.gz (GSMaP_MVK)
Daily Averaged Rain Data (p12Z-11Z averaged);
Directory: /standard/v8/daily/p12Z-11Z/YYYYMM/
File Name: gsmap_mvk.YYYYMMDD.0.1d.daily.p12Z-11Z.vP.RSKI.J.dat.gz (GSMaP_MVK)
Daily Averaged Gauge-calibrated Rain Data (00Z-23Z averaged);
Directory: /standard/v8/daily_G/00Z-23Z/YYYYMM/
File Name: gsmap_gauge.YYYYMMDD.0.1d.daily.00Z-23Z.vP.RSKI.J.dat.gz (GSMaP_Gauge)
Daily Averaged Gauge-calibrated Rain Data (p12Z-11Z averaged);
Directory: /standard/v8/daily_G/p12Z-11Z/YYYYMM/
File Name: gsmap_gauge.YYYYMMDD.0.1d.daily.p12Z-11Z.vP.RSKI.J.dat.gz (GSMaP_Gauge)
Hourly Rain & Gauge-calibrated Rain Data in Text Format;
Directory: /standard/v8/txt/hourly/XX_ZZZZZZ/YYYY/MM/DD/
File Name: gsmap_mvk_vPRSKIJ_YYYYMMDD_HH00_XX_ZZZZZZ.csv.zip (GSMaP_MVK & GSMaP_Gauge)
Daily Averaged Rain & Gauge-calibrated Rain Data in Text Format (00Z-23Z averaged);
Directory: /standard/v8/txt/daily/00Z-23Z/XX_ZZZZZZ/YYYY/MM/
File Name: gsmap_mvk_vPRSKIJ_YYYYMMDD_daily_00Z-23Z_XX_ZZZZZZ.csv.zip (GSMaP_MVK & GSMaP_Gauge)
Daily Averaged Rain & Gauge-calibrated Rain Data in Text Format (p12Z-11Z averaged);
Directory: /standard/v8/txt/daily/p12Z-11Z/XX_ZZZZZZ/YYYY/MM/
File Name: gsmap_mvk_vPRSKIJ_YYYYMMDD_daily_p12Z-11Z_XX_ZZZZZZ.csv.zip (GSMaP_MVK & GSMaP_Gauge)
Daily Averaged Rain & major flag in HDF;
Directory: /HDF/standard/v8/Daily/YYYY/MM
File Name: GPMMRG_MAP_YYMMDD_D_L3S_MCD_VVV.h5
Monthly Averaged Rain Data;
Directory: /standard/v8/monthly/YYYY/
File Name: gsmap_mvk.YYYYMM.0.1d.monthly.vP.RSKI.J.dat.gz (GSMaP_MVK)
Monthly Averaged Gauge-calibrated Rain Data;
Directory: /standard/v8/monthly_G/YYYY/
File Name: gsmap_gauge.YYYYMM.0.1d.monthly.vP.RSKI.J.dat.gz (GSMaP_Gauge)
Monthly Averaged Rain & Gauge-calibrated Rain Data in Text Format;
Directory: /standard/v8/txt/monthly/XX_ZZZZZZ/YYYY/
File Name: gsmap_mvk_vPRSKIJ_YYYYMM_monthly_XX_ZZZZZZ.csv.zip (GSMaP_MVK & GSMaP_Gauge)
Monthly Averaged Rain & major flag in HDF;
Directory: /HDF/standard/v8/Monthly/YYYY
File Name: GPMMRG_MAP_YYMM_M_L3S_MCM_VVV.h5
where,
YYYY: 4-digit year
YY: 2-digit year
MM: 2-digit month
DD: 2-digit day
HH: 2-digit hour
NN: 2-digit minute (currently fixed as 00)
P: Algorithm version
R: Version of microwave imager algorithm (reset when P is updated)
S: Version of microwave sounder algorithm (reset when P is updated)
K: Version of microwave imager/sounder algorithm (reset when P is updated)
I: Version of microwave-IR combined algorithm
J: Inclement number of reprocessing
XX_ZZZZZZ: 9-digit area name
VVV: 3-digit product version.
********************************************************************************
* Sample code
********************************************************************************
Directory: /standard/v8/sample/
Fortran Sample Code: read_GSMaP_MVK_0.1deg.v8.f
IDL Sample Code: GSMaP_MVK_sample.v8.pro
Python sample code: readGSMaP_MVK_netcdf.py
GrADS Control File:
Hourly Rainfall Data GSMaP_MVK.hourly.rain.v8.ctl
Satellite Information Flag GSMaP_MVK.hourly.sat.v8.ctl
Observation Time Flag GSMaP_MVK.hourly.time.v8.ctl
Reliability Flag GSMaP_MVK.hourly.reliability.v8.ctl
Daily (00Z-23Z avaraged) GSMaP_MVK.daily.00Z-23Z.v8.ctl
Daily (p12Z-11Z averaged) GSMaP_MVK.daily.p12Z-11Z.v8.ctl
Monthly GSMaP_MVK.monthly.v8.ctl
********************************************************************************
=== Directory contents of /standard/v8/monthly_G ===
drwxr-xr-x 2 411 trmm 4096 Sep 7 2023 1998
drwxr-xr-x 2 411 trmm 4096 Sep 7 2023 1999
drwxr-xr-x 2 411 trmm 4096 Sep 14 2023 2000
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2001
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2002
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2003
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2004
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2005
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2006
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2007
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2008
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2009
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2010
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2011
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2012
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2013
drwxr-xr-x 2 411 trmm 4096 Jul 9 2023 2014
drwxr-xr-x 2 411 trmm 4096 Sep 7 2023 2015
drwxr-xr-x 2 411 trmm 4096 Sep 7 2023 2016
drwxr-xr-x 2 411 trmm 4096 Sep 7 2023 2017
drwxr-xr-x 2 411 trmm 4096 Sep 7 2023 2018
drwxr-xr-x 2 411 trmm 4096 Sep 7 2023 2019
drwxr-xr-x 2 411 trmm 4096 Sep 7 2023 2020
drwxr-xr-x 2 411 trmm 4096 Sep 14 2023 2021
drwxr-xr-x 2 411 trmm 4096 Mar 13 2023 2022
drwxr-xr-x 2 411 trmm 4096 Jan 3 2024 2023
drwxr-xr-x 2 411 trmm 4096 Jan 3 2025 2024
drwxr-xr-x 2 411 trmm 4096 Jan 3 2026 2025
drwxr-xr-x 2 411 trmm 4096 Sep 3 03:56 2026
# ==============================================================================
# [Feature]
# JAXA GSMaP(monthly_G)のFTPサーバーから、過去10年間(2016〜2025年)の月間降水量データを
# 自動ダウンロードします。全120ヶ月分のデータ(mm/hr)を月ごとの日数を用いて総降水量に換算・累計し、
# 最後に10で割ることで「過去10年の平均年間降水量マップ」をGeoTIFFとして出力します。
#
# [License & Disclaimer]
# Copyright (c) 2026 [Author Name] / Released under the MIT License.
# 【免責事項】本スクリプトによるデータ取得および算出結果は衛星観測に基づく推計値であり、
# その正確性や特定目的への適合性を保証するものではありません。
# 開発者は本コードの利用により生じた一切の損害について責任を負いません。
# ==============================================================================
import ftplib
import os
import gzip
import calendar
import numpy as np
import rasterio
from rasterio.transform import from_origin
# FTP_HOST = "hokusai.eorc.jaxa.jp"
# FTP_USER = "YOUR_USERNAME"
# FTP_PASS = "YOUR_PASSWORD"
START_YEAR = 2016
END_YEAR = 2025 # 2016〜2025の10年間
BASE_DIR = "./gsmap_monthly_G_10years"
OUTPUT_TIFF = "gsmap_10yr_avg_annual_precipitation.tif"
def download_10years_data():
os.makedirs(BASE_DIR, exist_ok=True)
with ftplib.FTP(FTP_HOST) as ftp:
ftp.login(FTP_USER, FTP_PASS)
for year in range(START_YEAR, END_YEAR + 1):
ftp_dir = f"/standard/v8/monthly_G/{year}/"
try:
ftp.cwd(ftp_dir)
files = ftp.nlst("*.dat.gz")
for file in files:
local_path = os.path.join(BASE_DIR, file)
if not os.path.exists(local_path):
print(f"Downloading {file}...")
with open(local_path, "wb") as f:
ftp.retrbinary(f"RETR {file}", f.write)
except Exception as e:
print(f"Error accessing directory for {year}: {e}")
print("All downloads completed.")
def calculate_10yr_average_and_export():
rows, cols = 1200, 3600
total_cumulative = np.zeros((rows, cols), dtype=np.float32)
files = [f for f in os.listdir(BASE_DIR) if f.endswith(".dat.gz")]
for file in files:
# gsmap_gauge.YYYYMM.0.1d... から年と月を抽出
year_str = file.split(".")[1][0:4]
month_str = file.split(".")[1][4:6]
month_days = calendar.monthrange(int(year_str), int(month_str))[1]
filepath = os.path.join(BASE_DIR, file)
with gzip.open(filepath, 'rb') as f:
data = np.frombuffer(f.read(), dtype='<f4').reshape(2, rows, cols)[0]
data = np.where(data < 0, 0, data) # 欠損値処理
total_cumulative += data * 24 * month_days # mm/monthに換算して加算
# 10年分の総雨量を10で割り、年間平均降水量を算出
avg_annual = total_cumulative / (END_YEAR - START_YEAR + 1)
transform = from_origin(0.0, 60.0, 0.1, 0.1)
with rasterio.open(
OUTPUT_TIFF, 'w', driver='GTiff',
height=rows, width=cols, count=1, dtype=avg_annual.dtype,
crs='+proj=latlong +datum=WGS84', transform=transform, nodata=-9999
) as dst:
dst.write(avg_annual, 1)
print(f"Exported to {OUTPUT_TIFF}")
if __name__ == "__main__":
download_10years_data()
calculate_10yr_average_and_export()
これでPastTenYearsのRainFallのDataをGetしたうえで、10yearsのAnnualAvarageRainfallのGlobalなDistributionをProcess&Culculatできました。
However、ThisCodeのままだと、DataがGoogleColabのTemporaryFolderにSaveされ、GoogleColabをCloseするためにMakeDeleteするので、TemporaryFolderにSaveされたFileをMyGoogleDriveのDirectedFolderにTransferしてDataを永続化しておきました。
コメント
コメントを投稿