[weverse] add extractors #4768

bradenhilton · 2023-11-04T18:24:38Z

Closes #3995.

Weverse requires an account for paginated URLs. Its API is not publicly documented.

The API code is inspired by the yt-dlp implementation.

~~Speaking of yt-dlp, we could defer to it for media posts, as it supports VOD and live broadcasts as far as I can tell, but I haven't done this yet.~~

Some miscellaneous questions:

I noticed that you recently set Referer for requests by default, but it is absolutely mandatory for Weverse. Is it worth setting in the API headers?
The API responses use camelCase. Currently I am adding custom values to the metadata using snake_case. Should I stick to one style instead of mixing them?

bradenhilton · 2023-11-05T13:46:01Z

Example API responses:

Post (image)

{
  "extension": {},
  "body": "드디어 우리 노래 나왔잖아요!!\uD83D\uDC40\uD83E\uDEF6 \n피어나 어때요~~~??\n<w:attachment type=\"photo\" id=\"4-274880925\" />",
  "tags": [],
  "postId": "1-128752990",
  "postType": "NORMAL",
  "emotionCount": 30344,
  "commentCount": 3812,
  "publishedAt": 1698416087773,
  "membershipOnly": false,
  "availableActions": [
    "WRITE_COMMENT",
    "BOOKMARK",
    "WRITE_EMOTION",
    "READ_POST"
  ],
  "sectionType": "ARTIST",
  "hideFromArtist": false,
  "plainBody": "드디어 우리 노래 나왔잖아요!!\uD83D\uDC40\uD83E\uDEF6 \n피어나 어때요~~~??",
  "attachment": {
    "photo": {
      "4-274880925": {
        "url": "https://phinf.wevpstatic.net/MjAyMzEwMjdfMTMy/MDAxNjk4NDE2MDg3NDg0.uCJ26Yp7LLUBOJkCzex48s0BeG9Z8A4lgXZwWAqw64gg.c2KZ81NcEcFF4oJmGqpcuSH7MGAwwq9YkVIq3feLq04g.JPEG/image.jpg",
        "width": 2316,
        "height": 3088,
        "photoId": "4-274880925"
      }
    }
  },
  "shareUrl": "https://weverse.io/lesserafim/artist/1-128752990",
  "bookmarked": false,
  "viewerEmotionId": "3-1441778405",
  "locked": false,
  "hasProduct": false,
  "author": {
    "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
    "communityId": 47,
    "joined": true,
    "profileName": "쥬하\uD83C\uDF3F",
    "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
    "memberJoinStatus": "JOIN_COMMUNITY",
    "profileType": "ARTIST",
    "artistOfficialProfile": {
      "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
      "officialName": "KAZUHA",
      "birthday": {
        "date": 1060387200000,
        "dateString": "2003.08.09",
        "timezone": "Asia/Seoul",
        "bday": false
      }
    },
    "hasOfficialMark": true,
    "profileSpaceStatus": "ACCESSIBLE",
    "myProfile": false
  },
  "community": {
    "communityId": 47,
    "communityName": "LE SSERAFIM",
    "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
    "artistCode": "LESSERAFIM",
    "homeGradationColor": {
      "upLeftColorCode": "#7887c3",
      "upRightColorCode": "#455698",
      "downLeftColorCode": "#c0cbf5",
      "downRightColorCode": "#8e9edf",
      "menuTabLeftColorCode": "#465388",
      "menuTabRightColorCode": "#1b2e79"
    }
  }
}

Post (video)

{
  "extension": {},
  "body": "너무 귀엽죠??\uD83E\uDEF6\n오늘부터 잘 부탁해요~~\uD83D\uDE09 <w:attachment type=\"video\" id=\"4-864785\" />",
  "attachment": {
    "video": {
      "4-864785": {
        "videoId": "4-864785",
        "uploadInfo": {
          "width": 720,
          "videoId": "FC3D5A7233E610269DD0BB16F34AC9175D92",
          "playTime": 3,
          "height": 720,
          "imageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMDVfMjU5/MDAxNjk2NDg5MTg0NTg3.MqPrC4culB2vLsB1wjdKLSgzHuIrwNyfbZ2NFVxIXZAg.q5HU37Askl9whC3wZ9lMOymXanLgm68UypqZRYKH8Kgg.JPEG/bf494c1d-634c-11ee-9671-80615f0bcd6e_01.jpg",
          "uploadInfoVersion": 1
        }
      }
    }
  },
  "emotionCount": 28853,
  "commentCount": 2379,
  "postId": "2-127465849",
  "tags": [],
  "publishedAt": 1696489184386,
  "sectionType": "ARTIST",
  "membershipOnly": false,
  "availableActions": [
    "BOOKMARK",
    "WRITE_COMMENT",
    "WRITE_EMOTION",
    "READ_POST"
  ],
  "hideFromArtist": false,
  "postType": "NORMAL",
  "plainBody": "너무 귀엽죠??\uD83E\uDEF6\n오늘부터 잘 부탁해요~~\uD83D\uDE09",
  "shareUrl": "https://weverse.io/lesserafim/artist/2-127465849",
  "bookmarked": false,
  "locked": false,
  "hasProduct": false,
  "author": {
    "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
    "communityId": 47,
    "joined": true,
    "profileName": "쥬하\uD83C\uDF3F",
    "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
    "memberJoinStatus": "JOIN_COMMUNITY",
    "profileType": "ARTIST",
    "artistOfficialProfile": {
      "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
      "officialName": "KAZUHA",
      "birthday": {
        "date": 1060387200000,
        "dateString": "2003.08.09",
        "timezone": "Asia/Seoul",
        "bday": false
      }
    },
    "hasOfficialMark": true,
    "profileSpaceStatus": "ACCESSIBLE",
    "myProfile": false
  },
  "community": {
    "communityId": 47,
    "communityName": "LE SSERAFIM",
    "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
    "artistCode": "LESSERAFIM",
    "homeGradationColor": {
      "upLeftColorCode": "#7887c3",
      "upRightColorCode": "#455698",
      "downLeftColorCode": "#c0cbf5",
      "downRightColorCode": "#8e9edf",
      "menuTabLeftColorCode": "#465388",
      "menuTabRightColorCode": "#1b2e79"
    }
  }
}

Post (text)

{
  "extension": {},
  "tags": [],
  "publishedAt": 1696763700370,
  "emotionCount": 27997,
  "commentCount": 25772,
  "body": "피어나 뭐해요\uD83E\uDD79\uD83E\uDEF6",
  "availableActions": [
    "READ_POST",
    "BOOKMARK",
    "WRITE_COMMENT",
    "WRITE_EMOTION"
  ],
  "membershipOnly": false,
  "sectionType": "ARTIST",
  "plainBody": "피어나 뭐해요\uD83E\uDD79\uD83E\uDEF6",
  "postId": "3-135964325",
  "hideFromArtist": false,
  "attachment": {},
  "postType": "NORMAL",
  "shareUrl": "https://weverse.io/lesserafim/artist/3-135964325",
  "bookmarked": false,
  "locked": false,
  "hasProduct": false,
  "author": {
    "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
    "communityId": 47,
    "joined": true,
    "profileName": "쥬하\uD83C\uDF3F",
    "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
    "memberJoinStatus": "JOIN_COMMUNITY",
    "profileType": "ARTIST",
    "artistOfficialProfile": {
      "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
      "officialName": "KAZUHA",
      "birthday": {
        "date": 1060387200000,
        "dateString": "2003.08.09",
        "timezone": "Asia/Seoul",
        "bday": false
      }
    },
    "hasOfficialMark": true,
    "profileSpaceStatus": "ACCESSIBLE",
    "myProfile": false
  },
  "artistReactions": [
    {
      "reactionType": "comment",
      "reactionCount": 19,
      "member": {
        "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
        "communityId": 47,
        "joined": true,
        "profileName": "쥬하\uD83C\uDF3F",
        "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
        "memberJoinStatus": "JOIN_COMMUNITY",
        "profileType": "ARTIST",
        "artistOfficialProfile": {
          "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
          "officialName": "KAZUHA",
          "birthday": {
            "date": 1060387200000,
            "dateString": "2003.08.09",
            "timezone": "Asia/Seoul",
            "bday": false
          }
        },
        "hasOfficialMark": true,
        "profileSpaceStatus": "ACCESSIBLE",
        "myProfile": false
      }
    }
  ],
  "community": {
    "communityId": 47,
    "communityName": "LE SSERAFIM",
    "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
    "artistCode": "LESSERAFIM",
    "homeGradationColor": {
      "upLeftColorCode": "#7887c3",
      "upRightColorCode": "#455698",
      "downLeftColorCode": "#c0cbf5",
      "downRightColorCode": "#8e9edf",
      "menuTabLeftColorCode": "#465388",
      "menuTabRightColorCode": "#1b2e79"
    }
  }
}

Moment

{
  "extension": {
    "moment": {
      "momentAsJson": "{\"description\":\"\",\"style\":\"\",\"version\":\"1.1.3\",\"clips\":[{\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\",\"events\":[],\"title\":\"\",\"videoId\":\"940264D33932D0CD690A57AA820458275CD7\",\"duration\":10,\"metadata\":{\"texts\":[{\"range\":{\"endTime\":10000,\"startTime\":0},\"text\":\"한국에서 처음으로 보낸 추석\uD83E\uDD0D\\n피어나 여러분 뭐해요~~??\"}]},\"resolution\":{\"width\":1080,\"height\":1920}}],\"tags\":[],\"isReviewedByCustomer\":\"N\",\"title\":\"\",\"meta\":{\"stickerCnt\":{\"giphy\":0},\"mediaSrcCnt\":{\"video\":0,\"image\":1}},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\",\"duration\":10,\"indexes\":[{\"title\":\"\",\"range\":{\"endTime\":10000,\"startTime\":0},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\"}],\"resolution\":{\"width\":1080,\"height\":1920}}",
      "body": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
      "expireAt": 1662784496362,
      "video": {
        "videoId": "4-83503",
        "uploadInfo": {
          "width": 1080,
          "videoId": "940264D33932D0CD690A57AA820458275CD7",
          "playTime": 10,
          "height": 1920,
          "imageUrl": "https://phinf.wevpstatic.net/MjAyMjA5MDlfNTIg/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg",
          "uploadInfoVersion": 1
        }
      }
    }
  },
  "tags": [],
  "publishedAt": 1662698096362,
  "postType": "MOMENT",
  "sectionType": "MOMENT",
  "membershipOnly": false,
  "body": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
  "hideFromArtist": false,
  "emotionCount": 4490,
  "commentCount": 315,
  "availableActions": ["READ_POST"],
  "postId": "3-104040928",
  "attachment": {},
  "plainBody": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
  "shareUrl": "https://weverse.io/lesserafim/moment/b60d95bc3b71f4d97b28ac1b971cc641/post/3-104040928",
  "bookmarked": false,
  "locked": false,
  "hasProduct": false,
  "author": {
    "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
    "communityId": 47,
    "joined": true,
    "profileName": "쥬하\uD83C\uDF3F",
    "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
    "memberJoinStatus": "JOIN_COMMUNITY",
    "profileType": "ARTIST",
    "artistOfficialProfile": {
      "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
      "officialName": "KAZUHA",
      "birthday": {
        "date": 1060387200000,
        "dateString": "2003.08.09",
        "timezone": "Asia/Seoul",
        "bday": false
      }
    },
    "hasOfficialMark": true,
    "profileSpaceStatus": "ACCESSIBLE",
    "myProfile": false
  },
  "authorMomentPosts": {
    "paging": {},
    "data": [
      {
        "extension": {
          "moment": {
            "momentAsJson": "{\"description\":\"\",\"style\":\"\",\"version\":\"1.1.3\",\"clips\":[{\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\",\"events\":[],\"title\":\"\",\"videoId\":\"940264D33932D0CD690A57AA820458275CD7\",\"duration\":10,\"metadata\":{\"texts\":[{\"range\":{\"endTime\":10000,\"startTime\":0},\"text\":\"한국에서 처음으로 보낸 추석\uD83E\uDD0D\\n피어나 여러분 뭐해요~~??\"}]},\"resolution\":{\"width\":1080,\"height\":1920}}],\"tags\":[],\"isReviewedByCustomer\":\"N\",\"title\":\"\",\"meta\":{\"stickerCnt\":{\"giphy\":0},\"mediaSrcCnt\":{\"video\":0,\"image\":1}},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\",\"duration\":10,\"indexes\":[{\"title\":\"\",\"range\":{\"endTime\":10000,\"startTime\":0},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\"}],\"resolution\":{\"width\":1080,\"height\":1920}}",
            "body": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
            "expireAt": 1662784496362,
            "video": {
              "videoId": "4-83503",
              "uploadInfo": {
                "width": 1080,
                "videoId": "940264D33932D0CD690A57AA820458275CD7",
                "playTime": 10,
                "height": 1920,
                "imageUrl": "https://phinf.wevpstatic.net/MjAyMjA5MDlfNTIg/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg",
                "uploadInfoVersion": 1
              }
            }
          }
        },
        "tags": [],
        "publishedAt": 1662698096362,
        "postType": "MOMENT",
        "sectionType": "MOMENT",
        "membershipOnly": false,
        "hideFromArtist": false,
        "emotionCount": 4490,
        "commentCount": 315,
        "availableActions": ["READ_POST"],
        "postId": "3-104040928",
        "summary": {
          "videoCount": 1,
          "thumbnails": [
            {
              "type": "VIDEO",
              "url": "https://phinf.wevpstatic.net/MjAyMjA5MDlfNTIg/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg",
              "playTime": 10,
              "cVideoId": "4-83503"
            }
          ],
          "photoCount": 0,
          "snippets": []
        },
        "plainBody": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
        "shareUrl": "https://weverse.io/lesserafim/moment/b60d95bc3b71f4d97b28ac1b971cc641/post/3-104040928",
        "bookmarked": false,
        "locked": false,
        "hasProduct": false,
        "author": {
          "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
          "communityId": 47,
          "joined": true,
          "profileName": "쥬하\uD83C\uDF3F",
          "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
          "memberJoinStatus": "JOIN_COMMUNITY",
          "profileType": "ARTIST",
          "artistOfficialProfile": {
            "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
            "officialName": "KAZUHA",
            "birthday": {
              "date": 1060387200000,
              "dateString": "2003.08.09",
              "timezone": "Asia/Seoul",
              "bday": false
            }
          },
          "hasOfficialMark": true,
          "profileSpaceStatus": "ACCESSIBLE",
          "myProfile": false
        },
        "community": {
          "communityId": 47,
          "communityName": "LE SSERAFIM",
          "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
          "artistCode": "LESSERAFIM"
        }
      },
      {
        "extension": {
          "moment": {
            "momentAsJson": "{\"description\":\"\",\"style\":\"\",\"version\":\"1.1.3\",\"clips\":[{\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MjVfNzgg\\/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG\\/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg\",\"events\":[],\"title\":\"\",\"videoId\":\"45D6EC7EF41379A783252970FFAC04104648\",\"duration\":10,\"metadata\":{\"texts\":[{\"range\":{\"endTime\":10000,\"startTime\":0},\"text\":\"진짜 오늘 날씨 좋다~~~\uD83E\uDD0D\\n\\n저녁에도 밝은 여름 최고\uD83E\uDD1F\\n\\n좋은 하루 보내세요~~!!\"}]},\"resolution\":{\"width\":1080,\"height\":1920}}],\"tags\":[],\"isReviewedByCustomer\":\"N\",\"title\":\"\",\"meta\":{\"stickerCnt\":{\"giphy\":1},\"mediaSrcCnt\":{\"video\":0,\"image\":1}},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MjVfNzgg\\/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG\\/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg\",\"duration\":10,\"indexes\":[{\"title\":\"\",\"range\":{\"endTime\":10000,\"startTime\":0},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MjVfNzgg\\/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG\\/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg\"}],\"resolution\":{\"width\":1080,\"height\":1920}}",
            "body": "진짜 오늘 날씨 좋다~~~\uD83E\uDD0D\n\n저녁에도 밝은 여름 최고\uD83E\uDD1F\n\n좋은 하루 보내세요~~!!",
            "expireAt": 1658825391456,
            "video": {
              "videoId": "3-15501",
              "uploadInfo": {
                "width": 1080,
                "videoId": "45D6EC7EF41379A783252970FFAC04104648",
                "playTime": 10,
                "height": 1920,
                "imageUrl": "https://phinf.wevpstatic.net/MjAyMjA3MjVfNzgg/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg",
                "uploadInfoVersion": 1
              }
            }
          }
        },
        "tags": [],
        "publishedAt": 1658738991456,
        "postType": "MOMENT",
        "sectionType": "MOMENT",
        "membershipOnly": false,
        "hideFromArtist": false,
        "emotionCount": 6976,
        "commentCount": 468,
        "availableActions": ["READ_POST"],
        "postId": "4-101115099",
        "summary": {
          "videoCount": 1,
          "thumbnails": [
            {
              "type": "VIDEO",
              "url": "https://phinf.wevpstatic.net/MjAyMjA3MjVfNzgg/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg",
              "playTime": 10,
              "cVideoId": "3-15501"
            }
          ],
          "photoCount": 0,
          "snippets": []
        },
        "plainBody": "진짜 오늘 날씨 좋다~~~\uD83E\uDD0D\n저녁에도 밝은 여름 최고\uD83E\uDD1F\n좋은 하루 보내세요~~!!",
        "shareUrl": "https://weverse.io/lesserafim/moment/b60d95bc3b71f4d97b28ac1b971cc641/post/4-101115099",
        "bookmarked": false,
        "locked": false,
        "hasProduct": false,
        "author": {
          "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
          "communityId": 47,
          "joined": true,
          "profileName": "쥬하\uD83C\uDF3F",
          "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
          "memberJoinStatus": "JOIN_COMMUNITY",
          "profileType": "ARTIST",
          "artistOfficialProfile": {
            "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
            "officialName": "KAZUHA",
            "birthday": {
              "date": 1060387200000,
              "dateString": "2003.08.09",
              "timezone": "Asia/Seoul",
              "bday": false
            }
          },
          "hasOfficialMark": true,
          "profileSpaceStatus": "ACCESSIBLE",
          "myProfile": false
        },
        "community": {
          "communityId": 47,
          "communityName": "LE SSERAFIM",
          "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
          "artistCode": "LESSERAFIM"
        }
      },
      {
        "extension": {
          "moment": {
            "momentAsJson": "{\"description\":\"\",\"style\":\"\",\"version\":\"1.1.3\",\"clips\":[{\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MTlfMjg1\\/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG\\/95CE3178-E10B-4298-B877-396268E6798F.jpg\",\"events\":[],\"title\":\"\",\"videoId\":\"233B24713027EC3B674EDDE9FB61461CF404\",\"duration\":10,\"metadata\":{\"texts\":[{\"range\":{\"endTime\":10000,\"startTime\":0},\"text\":\"우와우\\n이런 것 있었는지 몰랐어!!ㅋㅋㅋㅋ \\n더 생기면 좋겠다\uD83E\uDEF6ㅋㅋㅋ\"}]},\"resolution\":{\"width\":1080,\"height\":1920}}],\"tags\":[],\"isReviewedByCustomer\":\"N\",\"title\":\"\",\"meta\":{\"stickerCnt\":{\"giphy\":3},\"mediaSrcCnt\":{\"video\":0,\"image\":1}},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MTlfMjg1\\/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG\\/95CE3178-E10B-4298-B877-396268E6798F.jpg\",\"duration\":10,\"indexes\":[{\"title\":\"\",\"range\":{\"endTime\":10000,\"startTime\":0},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MTlfMjg1\\/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG\\/95CE3178-E10B-4298-B877-396268E6798F.jpg\"}],\"resolution\":{\"width\":1080,\"height\":1920}}",
            "body": "우와우\n이런 것 있었는지 몰랐어!!ㅋㅋㅋㅋ \n더 생기면 좋겠다\uD83E\uDEF6ㅋㅋㅋ",
            "expireAt": 1658277011392,
            "video": {
              "videoId": "3-4450",
              "uploadInfo": {
                "width": 1080,
                "videoId": "233B24713027EC3B674EDDE9FB61461CF404",
                "playTime": 10,
                "height": 1920,
                "imageUrl": "https://phinf.wevpstatic.net/MjAyMjA3MTlfMjg1/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG/95CE3178-E10B-4298-B877-396268E6798F.jpg",
                "uploadInfoVersion": 1
              }
            }
          }
        },
        "tags": [],
        "publishedAt": 1658190611392,
        "postType": "MOMENT",
        "sectionType": "MOMENT",
        "membershipOnly": false,
        "hideFromArtist": false,
        "emotionCount": 4157,
        "commentCount": 252,
        "availableActions": ["READ_POST"],
        "postId": "4-100700103",
        "summary": {
          "videoCount": 1,
          "thumbnails": [
            {
              "type": "VIDEO",
              "url": "https://phinf.wevpstatic.net/MjAyMjA3MTlfMjg1/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG/95CE3178-E10B-4298-B877-396268E6798F.jpg",
              "playTime": 10,
              "cVideoId": "3-4450"
            }
          ],
          "photoCount": 0,
          "snippets": []
        },
        "plainBody": "우와우\n이런 것 있었는지 몰랐어!!ㅋㅋㅋㅋ \n더 생기면 좋겠다\uD83E\uDEF6ㅋㅋㅋ",
        "shareUrl": "https://weverse.io/lesserafim/moment/b60d95bc3b71f4d97b28ac1b971cc641/post/4-100700103",
        "bookmarked": false,
        "locked": false,
        "hasProduct": false,
        "author": {
          "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
          "communityId": 47,
          "joined": true,
          "profileName": "쥬하\uD83C\uDF3F",
          "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
          "memberJoinStatus": "JOIN_COMMUNITY",
          "profileType": "ARTIST",
          "artistOfficialProfile": {
            "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
            "officialName": "KAZUHA",
            "birthday": {
              "date": 1060387200000,
              "dateString": "2003.08.09",
              "timezone": "Asia/Seoul",
              "bday": false
            }
          },
          "hasOfficialMark": true,
          "profileSpaceStatus": "ACCESSIBLE",
          "myProfile": false
        },
        "community": {
          "communityId": 47,
          "communityName": "LE SSERAFIM",
          "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
          "artistCode": "LESSERAFIM"
        }
      }
    ]
  },
  "community": {
    "communityId": 47,
    "communityName": "LE SSERAFIM",
    "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
    "artistCode": "LESSERAFIM",
    "homeGradationColor": {
      "upLeftColorCode": "#7887c3",
      "upRightColorCode": "#455698",
      "downLeftColorCode": "#c0cbf5",
      "downRightColorCode": "#8e9edf",
      "menuTabLeftColorCode": "#465388",
      "menuTabRightColorCode": "#1b2e79"
    }
  }
}

For endpoints with pagination support, their response includes a paging.nextParams.after value that can be sent as an after param in the next request.

Some of these values could probably be removed from the final metadata (or the metadata could be created from scratch with cherry picked values), though I'm hesitant to remove too much for versatility with naming/filtering.

bradenhilton · 2023-11-05T22:56:50Z

I found myself needing to pull an all-nighter, so I went ahead and just implemented most of the TODOs to keep myself busy.

Some of the extractors probably aren't necessary. WeverseMomentsExtractor in particular could probably be merged into WeverseMomentExtractor, and an optional capture group could be used for single Moments instead. I wanted to keep things separate for now as I'm still learning how everything works.

bradenhilton · 2023-11-12T01:07:39Z

I found a new endpoint which was the missing piece to allow all non-paginating extractors to work without authentication. I have now added some more robust tests.

bradenhilton · 2024-02-11T19:41:50Z

@mikf I believe this is pretty much finished now (pending a review at your convenience, of course).

I attempted to rewrite it in the style of some other extractors such as twitter and bluesky (e.g. using something like posts() instead of items() where possible), but I'm unsure if there are enough similarities between Weverse and those sites for it to be of any use, so I reverted to delegating for all extractors that use pagination.

ytdl support (for non-embed videos) has been removed. Only yt-dlp has Weverse extractors (for Media videos and live streams/VODs, not Post videos), I am not sure if ytdl support adds much value here. ytdl is still used if embeds is enabled.

I also have most if not all possible relevant API responses saved if you'd like them for context (or to create the extractors yourself if you prefer).

Hrxn · 2024-05-31T14:27:38Z

@mikf

bradenhilton force-pushed the weverse branch from 4b512b6 to 3b60dab Compare November 4, 2023 18:40

bradenhilton force-pushed the weverse branch from 3b60dab to 3dfc3ce Compare November 5, 2023 22:55

bradenhilton force-pushed the weverse branch 2 times, most recently from cbb6bc7 to 6f16e1a Compare November 7, 2023 10:13

bradenhilton marked this pull request as ready for review November 7, 2023 10:14

This comment was marked as outdated.

Sign in to view

bradenhilton force-pushed the weverse branch 3 times, most recently from 55c8167 to 7954d84 Compare November 12, 2023 01:02

bradenhilton force-pushed the weverse branch from 7954d84 to fff03a4 Compare November 13, 2023 01:47

bradenhilton force-pushed the weverse branch 6 times, most recently from a2ad565 to 963d311 Compare February 11, 2024 02:55

[weverse] add extractors

ff661fa

bradenhilton force-pushed the weverse branch from 963d311 to ff661fa Compare May 31, 2024 12:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[weverse] add extractors #4768

[weverse] add extractors #4768

bradenhilton commented Nov 4, 2023 •

edited

Loading

bradenhilton commented Nov 5, 2023

bradenhilton commented Nov 5, 2023

This comment was marked as outdated.

bradenhilton commented Nov 12, 2023

bradenhilton commented Feb 11, 2024 •

edited

Loading

Hrxn commented May 31, 2024

[weverse] add extractors #4768

Are you sure you want to change the base?

[weverse] add extractors #4768

Conversation

bradenhilton commented Nov 4, 2023 • edited Loading

bradenhilton commented Nov 5, 2023

bradenhilton commented Nov 5, 2023

This comment was marked as outdated.

bradenhilton commented Nov 12, 2023

bradenhilton commented Feb 11, 2024 • edited Loading

Hrxn commented May 31, 2024

bradenhilton commented Nov 4, 2023 •

edited

Loading

bradenhilton commented Feb 11, 2024 •

edited

Loading