大家沉迷短视频无法自拔？Python爬虫进阶，带你玩转短视频

大家好，我是辣条。

现在短视频可谓是一骑绝尘，吃饭的时候、休息的时候、躺在床上都在刷短视频，今天给大家带来python爬虫进阶 ：美拍视频地址加密解析。

短视频js逆向解析

抓取目标

工具使用

开发环境：win10、python3.7
开发工具：pycharm、Chrome
工具包：requests、xpath、base64
重点学习内容

爬虫采集数据的解析过程
js代码调试技巧
js逆向解析代码
Python代码的转换
项目思路解析

进入到网站的首页
挑选你感兴趣的分类根据首页地址获取到进入详情页面的超链接的跳转地址

    # eplace()方法用于在字符串中用一些字符替换另一些字符    # parseInt 数据转换成对应的整型    # base64.atob   对base64编码过的字符串进行解码    # substring 方法可在字符串中抽取从 start 下标开始的指定数目的字符

将js代码转换成Python代码

import base64def decode(data):    def getHex(a):        return {            'str': a[4:],            'hex': ''.join(list(a[:4])[::-1]),        }    def getDec(a):        b = str(int(a, 16))        return {            'pre': list(b[:2]),            'tail': list(b[2:]),        }    def substr(a, b):        c = a[0: int(b[0])]        d = a[int(b[0]): int(b[0]) + int(b[1])]        return c + a[int(b[0]):].replace(d, "")    def getPos(a, b):        b[0] = len(a) - int(b[0]) - int(b[1])        return b    b = getHex(data)    c = getDec(b['hex'])    d = substr(b['str'], c['pre'])    return base64.b64decode(substr(d, getPos(d, c['tail'])))print(decode("e121Ly9tBrI84RdnZpZGVvMTAubWVpdHVkYXRhLmNvbS82MGJjZDcwNTE3NGZieXBueG5udnRwMTA5N19IMjY0XzFfNWY3YThmM2U0MTEwNy5tc2JVjAu3EDQ="))

简易源码分享

import requestsfrom lxml import etreeimport base64def decode_mp4(data):    def getHex(a):        return {            'str': a[4:],            'hex': ''.join(list(a[:4])[::-1]),        }    def getDec(a):        b = str(int(a, 16))        return {            'pre': list(b[:2]),            'tail': list(b[2:]),        }    def substr(a, b):        c = a[0: int(b[0])]        d = a[int(b[0]): int(b[0]) + int(b[1])]        return c + a[int(b[0]):].replace(d, "")    def getPos(a, b):        b[0] = len(a) - int(b[0]) - int(b[1])        return b    b = getHex(data)    c = getDec(b['hex'])    d = substr(b['str'], c['pre'])    return base64.b64decode(substr(d, getPos(d, c['tail'])))# 运行主函数def main():    url = 'https://www.meipai.com'    headers = {        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',    }    response = requests.get(url=url, headers=headers)    html_data = etree.HTML(response.text)    href_list = html_data.xpath('//div/a/@href')    # print(href_list)    for href in href_list:        res = requests.get('https://www.meipai.com' + href, headers=headers)        html = etree.HTML(res.text)        name = html.xpath('//div[@id="detailVideo"]/img/@alt')[0]        mp4_data = html.xpath('//div[@id="detailVideo"]/@data-video')[0]        # print(name, mp4_data)        mp4_url = decode_mp4(mp4_data).decode('utf-8')        print(mp4_url)        result = requests.get("http:" + mp4_url)        with open(name + ".mp4", 'wb') as f:            f.write(result.content)            f.close()if __name__ == '__main__':    main()