修正一些逻辑细节

This commit is contained in:
2021-05-04 20:31:37 +08:00
parent 630c36983a
commit 79a5f3a2d5
5 changed files with 159 additions and 188 deletions
-51
View File
@@ -1,51 +0,0 @@
# wxbot
微信聊天记录自动存sqlite数据库,基于微信网页版。注意,Web版获取不到微信用户的微信ID,所得到的用户名/ID是仅当前会话有效的,因此只有根据备注名在数据库中识别用户比较靠谱。
项目fork自[@Urinx/WeixinBot](https://github.com/Urinx/WeixinBot),我只是加了个存到sqlite的壳,感谢老哥的网页版微信API。
## 数据表结构
| 字段名 | 类型 | 说明 |
| -- | -- | -- |
| msg_id| TEXT | 消息ID |
|create_time | INTEGER | 创建时间,Unix时间戳 |
| msg_type | INTEGER | 消息类型,1-文本,3-图片,34-语音,43-视频,0-其它
| content | TEXT | 消息的文本内容
| from_id | TEXT | 消息发送者ID(不是微信ID,没啥用),如果是来自群的消息此字段为空
| from_name | TEXT | 消息发送者的备注,没备注就昵称,如果是来自群的消息此字段为空
| from_nickname | TEXT | 消息发送者的昵称,如果是来自群的消息此字段为空
| to_id | TEXT | 接收者ID,如果是发送到群的消息此字段为空
| to_name | TEXT | 接收者备注,没备注就昵称,如果是发送到群的消息此字段为空
| to_nickname | TEXT | 接收者昵称,如果是发送到群的消息此字段为空
| group_id | TEXT | 如果是群消息则为群的ID否则为空,照样,没啥用
| group_name | TEXT | 如果是群消息则为群聊名称(应该吧🤣我不关注群消息),否则为空
消息里的图片等文件以`img_``voice_`等前缀 + 消息ID存储于命令行参数给定的数据目录中。
其它信息请参考[项目@Urinx/WeixinBot](https://github.com/Urinx/WeixinBot)。
## 使用
### 克隆仓库并安装python依赖
```shell
# 不适用win系统
git clone https://github.com/mattuylee/wxbot.git
cd wxbot
python3 -m venv ./venv
source ./venv/bin/activate
pip3 install -r ./requirements.txt
```
### 运行
`python ./main.py -f <DB_FILE> -d <DATA_DIR>`
其中`DB_FILE`是sqlite数据库文件路径,`DATA_DIR`是图片视频等数据的存储目录。
支持sqlcipher加密,程序启动会要求输入密码,如不需要加密可留空直接回车。注意,以后每次启动必须输入相同的密码。暂不支持修改密码,可自行安装sqlcipher打开数据库修改密码。
未加密数据库可直接通过sqlite3打开,加密数据库需要sqlcipher才能打开。程序只负责记录数据,没有读取能力。
### 其它
关于表情包,如果是从表情商店里添加的表情包网页版微信是获取不到的,为了保护创作者的权益。这倒是可以理解。
如果发现无法登录先试试在浏览器登录网页版微信,有可能是腾讯为了“安全”限制了你的微信帐号在网页登录。
最后,祝腾讯早点倒闭。
+12 -12
View File
@@ -7,20 +7,20 @@
## 数据表结构
| 字段名 | 类型 | 说明 |
| -- | -- | -- |
| msg_id| TEXT | 消息ID |
| msg_id| TEXT | 消息ID,结构为创建时间+原微信消息ID |
|create_time | INTEGER | 创建时间,Unix时间戳 |
| msg_type | INTEGER | 消息类型,1-文本,3-图片,34-语音,43-视频,0-其它
| content | TEXT | 消息的文本内容
| from_id | TEXT | 消息发送者ID(不是微信ID,没啥用),如果是来自群的消息此字段为空
| from_name | TEXT | 消息发送者的备注,没备注就昵称,如果是来自群的消息此字段为空
| from_nickname | TEXT | 消息发送者的昵称,如果是来自群的消息此字段为空
| to_id | TEXT | 接收者ID,如果是发送到群的消息此字段为空
| to_name | TEXT | 接收者备注,没备注就昵称,如果是发送到群的消息此字段为空
| to_nickname | TEXT | 接收者昵称,如果是发送到群的消息此字段为空
| group_id | TEXT | 如果是群消息则为群的ID否则为空,照样,没啥用
| group_name | TEXT | 如果是群消息则为群聊名称(应该吧🤣我不关注群消息),否则为空
| msg_type | INTEGER | 消息类型,1-文本,3-图片,34-语音,43-视频,0-其它 |
| content | TEXT | 消息的文本内容 |
| from_id | TEXT | 消息发送者ID(不是微信ID,没啥用),如果是来自群的消息此字段为空 |
| from_name | TEXT | 消息发送者的备注,没备注就昵称,如果是来自群的消息此字段为空 |
| from_nickname | TEXT | 消息发送者的昵称,如果是来自群的消息此字段为空 |
| to_id | TEXT | 接收者ID,如果是发送到群的消息此字段为空 |
| to_name | TEXT | 接收者备注,没备注就昵称,如果是发送到群的消息此字段为空 |
| to_nickname | TEXT | 接收者昵称,如果是发送到群的消息此字段为空 |
| group_id | TEXT | 如果是群消息则为群的ID否则为空,照样,没啥用 |
| group_name | TEXT | 如果是群消息则为群聊名称(应该吧🤣我不关注群消息),否则为空 |
消息里的图片等文件以`img_``voice_`等前缀 + 消息ID存储于命令行参数给定的数据目录中。
消息里的图片等文件以消息ID存储于命令行参数给定的数据目录中,并根据年份划分子目录
其它信息请参考[项目@Urinx/WeixinBot](https://github.com/Urinx/WeixinBot)。
+99 -62
View File
@@ -1,7 +1,10 @@
#!/usr/bin/env python3
# coding: utf-8
import argparse
import getpass
import logging
import sys
import time
import os
from weixin import WebWeixin
from sqlcipher3 import dbapi2 as sqlite
@@ -9,8 +12,9 @@ from sqlcipher3 import dbapi2 as sqlite
def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--db-file", required=True, help="数据库文件")
parser.add_argument("-d", "--data-dir",required=True, help="文件存储目录")
parser.add_argument('-f', '--db-file', required=True, help='数据库文件')
parser.add_argument('-d', '--data-dir',required=True, help='文件存储目录')
parser.add_argument('--debug', action='store_true', help='调试模式')
args = parser.parse_args()
return args
@@ -24,69 +28,116 @@ class WXBot(WebWeixin):
return member['NickName']
return None
def composeMessageID(self, msgID, timestamp):
return time.strftime('%Y%m%d%H%M%S', time.localtime(timestamp)) + msgID
def saveMessageImage(self, msgid, composedID):
url = self.base_uri + \
'/webwxgetmsgimg?MsgID=%s&skey=%s' % (msgid, self.skey)
data = self._get(url, 'webwxgetmsgimg')
return self.saveFile(composedID[:4], composedID + '.jpg', data)
def saveVoice(self, msgid, composedID):
url = self.base_uri + \
'/webwxgetvoice?msgid=%s&skey=%s' % (msgid, self.skey)
data = self._get(url, api='webwxgetvoice')
return self.saveFile(composedID[:4], composedID + '.mp3', data)
def saveVideo(self, msgid, composedID):
url = self.base_uri + \
'/webwxgetvideo?msgid=%s&skey=%s' % (msgid, self.skey)
data = self._get(url, api='webwxgetvideo')
return self.saveFile(composedID[:4], composedID + '.mp4', data)
def saveFile(self, subdir, filename, data):
if data == '':
return False
dirName = os.path.join(dataDir, subdir)
if not os.path.exists(dirName):
os.makedirs(dirName)
fn = os.path.join(dirName, filename)
with open(fn, 'wb') as f:
f.write(data)
f.close()
return True
def handleMsg(self, r):
if self.DEBUG:
print("处理消息, NewMessageCount =", r['AddMsgCount'])
for msg in r['AddMsgList']:
msgID = msg['MsgId']
composedID = self.composeMessageID(msgID, msg['CreateTime'])
msgType = msg['MsgType']
if msg['FromUserName'] in self.SpecialUsers or msg['ToUserName'] in self.SpecialUsers:
# 过滤特殊帐号消息
continue
elif msgType in (50, 51, 52, 53, 9999):
#无用消息
continue
sql = "INSERT INTO msg (\
msg_id,\
create_time,\
msg_type,\
content,\
from_id,\
from_name,\
from_nickname,\
to_id,\
to_name,\
to_nickname,\
group_id,\
group_name\
) VALUES (\
:msg_id,\
:create_time,\
:msg_type,\
:content,\
:from_id,\
:from_name,\
:from_nickname,\
:to_id,\
:to_name,\
:to_nickname,\
:group_id,\
:group_name\
)"
params = {
"msg_id": msgID,
"msg_id": composedID,
"create_time": msg['CreateTime'],
"msg_type": msgType,
"content": None,
"from_id": msg['FromUserName'],
"from_name": self.getUserRemarkName(msg['FromUserName']),
"from_nickname": self.getUserNickName(msg['FromUserName']),
"to_id": msg['ToUserName'],
"to_name": self.getUserRemarkName(msg['ToUserName']),
"to_nickname": self.getUserNickName(msg['ToUserName']),
"group_id": None,
"group_name": None
}
# 过滤特殊帐号消息
if msg['FromUserName'] in self.SpecialUsers or msg['ToUserName'] in self.SpecialUsers:
continue
if msg['FromUserName'] == self.User['UserName']:
params['from_name'] = '_self'
elif msg['ToUserName'] == self.User['UserName']:
params['to_name'] = '_self'
if msg['FromUserName'][:2] == '@@':
# 群消息
if ":<br/>" in content:
[people, content] = msg['Content'].split(':<br/>', 1)
params['group_name'] = self.getUserRemarkName(msg['FromUserName'])
params['from_name'] = self.getUserRemarkName(people)
params['to_name'] = params['to_nickname'] = None
elif msg['ToUserName'][:2] == '@@':
# 发送的群消息
params['group_name'] = self.getUserRemarkName(msg['ToUserName'])
params['from_name'] = self.User['NickName']
params['to_name'] = params['to_nickname'] = None
if msgType == 1:
# 文字消息
params['content'] = msg['Content'].replace('&lt;', '<').replace('&gt;', '>')
elif msgType == 3:
params['content'] = "[图片](%s)" % msgID
self.webwxgetmsgimg(msgID)
params['content'] = "[图片](%s)" % composedID
self.saveMessageImage(msgID, composedID)
elif msgType == 34:
params['content'] = "[语音](%s)" % msgID
self.webwxgetvoice(msgID)
params['content'] = "[语音](%s)" % composedID
self.saveVoice(msgID, composedID)
elif msgType == 43:
params['content'] = "[视频](%s)" % msgID
self.webwxgetvideo(msgID)
params['content'] = "[视频](%s)" % composedID
self.saveVideo(msgID, composedID)
elif msgType == 62:
params['msg_type'] = 43
params['content'] = "[小视频](%s)" % msgID
self.webwxgetvideo(msgID)
params['content'] = "[小视频](%s)" % composedID
self.saveVideo(msgID, composedID)
elif msgType == 42:
params['msg_type'] = 0
params['content'] = "[名片](%s)" % msg['RecommendInfo']['NickName']
@@ -97,45 +148,43 @@ class WXBot(WebWeixin):
params['content'] = "[表情包图片]"
else:
params['msg_type'] = 3
params['content'] = "[动画表情](%s)" % msgID
self.webwxgetmsgimg(msgID)
params['content'] = "[动画表情](%s)" % composedID
self.saveMessageImage(msgID, composedID)
elif msgType == 49:
params['msg_type'] = 0
params['content'] = "[链接:%s](%s)" % (msg['FileName'], msg['Url'])
elif (50, 51, 52, 53, 9999, 10000, 10002).index(msgType) >= 0:
continue
elif msgType == 10000:
params['msg_type'] = 0
params['content'] = "[系统消息](%s)" % (msg['Content'])
elif msgType == 10002:
params['msg_type'] = 0
params['content'] = "[系统消息](%s撤回了一条消息)" % params['from_name']
else:
params['msg_type'] = 0
params['content'] = ""
continue
# 群消息
if msg['FromUserName'][:2] == '@@':
if ":<br/>" in content:
[people, content] = msg['Content'].split(':<br/>', 1)
params['group_name'] = self.getUserRemarkName(msg['FromUserName'])
params['from_id'] = people
params['from_name'] = self.getUserRemarkName(people)
params['to_id'] = params['to_name'] = params['to_nickname'] = None
elif msg['ToUserName'][:2] == '@@':
params['group_name'] = self.getUserRemarkName(msg['ToUserName'])
params['from_id'] = self.User['UserName']
params['from_name'] = self.User['NickName']
params['to_id'] = params['to_name'] = params['to_nickname'] = None
params['content'] = "[未识别消息]" + msg['Content']
conn.execute(sql, params)
conn.commit()
args = parse_args()
dataDir = args.data_dir
if not os.access(os.path.dirname(args.db_file), os.X_OK) and not os.access(args.db_file, os.W_OK):
if __name__ == '__main__':
if sys.stdout.encoding == 'cp936':
sys.stdout = UnicodeStreamFilter(sys.stdout)
logger = logging.getLogger(__name__)
if not sys.platform.startswith('win'):
import coloredlogs
coloredlogs.install(level='INFO')
args = parse_args()
dataDir = args.data_dir
if not os.access(os.path.dirname(args.db_file), os.X_OK) and not os.access(args.db_file, os.W_OK):
print("无法创建或打开数据库文件")
exit()
elif not os.access(dataDir, os.X_OK):
elif not os.access(dataDir, os.X_OK):
print("数据目录不存在或权限不足")
exit()
conn = sqlite.connect(args.db_file)
while True:
conn = sqlite.connect(args.db_file)
while True:
try:
passwd = getpass.getpass("输入数据库密码:")
if passwd != '':
@@ -145,13 +194,10 @@ while True:
create_time INTEGER NOT NULL,\
msg_type INTEGER NOT NULL,\
content TEXT,\
from_id TEXT,\
from_name TEXT,\
from_nickname TEXT,\
to_id TEXT,\
to_name TEXT,\
to_nickname TEXT,\
group_id TEXT,\
group_name TEXT\
);")
break
@@ -165,16 +211,7 @@ while True:
except Exception as e:
print("打开数据库失败", e)
exit(1)
if sys.stdout.encoding == 'cp936':
sys.stdout = UnicodeStreamFilter(sys.stdout)
if __name__ == '__main__':
logger = logging.getLogger(__name__)
if not sys.platform.startswith('win'):
import coloredlogs
coloredlogs.install(level='INFO')
wxbot = WXBot()
wxbot.DEBUG = args.debug
wxbot.saveFolder = dataDir
wxbot.start()
+21 -18
View File
@@ -118,7 +118,6 @@ class WebWeixin(object):
self.memberCount = 0
self.SpecialUsers = ['newsapp', 'fmessage', 'filehelper', 'weibo', 'qqmail', 'fmessage', 'tmessage', 'qmessage', 'qqsync', 'floatbottle', 'lbsapp', 'shakeapp', 'medianote', 'qqfriend', 'readerapp', 'blogapp', 'facebookapp', 'masssendapp', 'meishiapp', 'feedsapp',
'voip', 'blogappweixin', 'weixin', 'brandsessionholder', 'weixinreminder', 'wxid_novlwrv3lqwv11', 'gh_22b87fa7cb3c', 'officialaccounts', 'notification_messages', 'wxid_novlwrv3lqwv11', 'gh_22b87fa7cb3c', 'wxitil', 'userexperience_alarm', 'notification_messages']
self.TimeOut = 20 # 同步最短时间间隔(单位:秒)
self.media_count = -1
self.cookie = http.cookiejar.CookieJar()
@@ -265,6 +264,15 @@ class WebWeixin(object):
}
return True
def logout(self):
request = urllib.request.Request(
url='https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxlogout?redirect=0&type=1&skey=%s' % self.skey,
data=urllib.parse.urlencode({
'sid': self.sid,
'uin': self.uin,
}).encode(encoding='utf-8'))
urllib.request.urlopen(request)
def webwxinit(self):
url = self.base_uri + '/webwxinit?pass_ticket=%s&skey=%s&r=%s' % (
self.pass_ticket, self.skey, int(time.time()))
@@ -406,7 +414,7 @@ class WebWeixin(object):
'_': int(time.time()),
}
url = 'https://' + self.syncHost + '/cgi-bin/mmwebwx-bin/synccheck?' + urllib.parse.urlencode(params)
data = self._get(url, timeout=5)
data = self._get(url, timeout=30)
if data == '':
return [-1,-1]
@@ -893,24 +901,18 @@ class WebWeixin(object):
logging.debug('[*] 你在其他地方登录了 WEB 版微信,债见')
break
elif retcode == '0':
if selector == '2':
shouldSleep = True
if selector != '0':
# 有需要同步的消息
r = self.webwxsync()
if r is not None:
if r is not None and r['AddMsgCount'] > 0:
shouldSleep = False
self.handleMsg(r)
elif selector == '6':
# TODO
redEnvelope += 1
print('[*] 收到疑似红包消息 %d' % redEnvelope)
logging.debug('[*] 收到疑似红包消息 %d' % redEnvelope)
elif selector == '7':
playWeChat += 1
print('[*] 你在手机上玩微信被我发现了 %d' % playWeChat)
logging.debug('[*] 你在手机上玩微信被我发现了 %d' % playWeChat)
r = self.webwxsync()
elif selector == '0':
time.sleep(1)
if (time.time() - self.lastCheckTs) <= 20:
time.sleep(time.time() - self.lastCheckTs)
if shouldSleep and time.time() - self.lastCheckTs <= 5:
# 微信有bug,发送某些消息后轮询会立马返回并报告有消息,但是调用同步接口后发现根本没有新消息,当收到其它消息后又会恢复
# 这个现象一般出现在发送表情包之后。可以在浏览器打开控制台,会发现在手机发送表情后轮询接口被疯狂调用
# 这里是防止轮询接口调用太频繁而实际上根本没有消息,浪费带宽
time.sleep(5)
def sendMsg(self, name, word, isfile=False):
id = self.getUSerID(name)
@@ -1020,6 +1022,7 @@ class WebWeixin(object):
listenProcess.terminate()
print('[*] 退出微信')
logging.debug('[*] 退出微信')
self.logout()
exit()
elif text[:2] == '->':
[name, word] = text[2:].split(':')
+2 -20
View File
@@ -1,8 +1,8 @@
{
"modules": [
{
"name": "core",
"chnname": "NOTA核心",
"name": "wxbot",
"chnname": "wxbot",
"entities": [
{
"title": "msg",
@@ -37,12 +37,6 @@
"remark": "",
"chnname": "消息内容"
},
{
"name": "from_id",
"type": "TEXT_ID",
"remark": "",
"chnname": "发送者ID"
},
{
"name": "from_name",
"type": "TEXT",
@@ -55,12 +49,6 @@
"remark": "昵称",
"chnname": "发送者昵称"
},
{
"name": "to_id",
"type": "TEXT_ID",
"remark": "",
"chnname": "接收者ID"
},
{
"name": "to_name",
"type": "TEXT",
@@ -73,12 +61,6 @@
"remark": "",
"chnname": "接收者昵称"
},
{
"name": "group_id",
"type": "TEXT_ID",
"remark": "",
"chnname": "群组ID"
},
{
"name": "group_name",
"type": "TEXT",